None
**Instruments Affected**: NIRSpec
Tested on CV3 data
# Create a temporary directory to hold notebook output, and change the working directory to that directory.
from tempfile import TemporaryDirectory
import os
import shutil
data_dir = TemporaryDirectory()
os.chdir(data_dir.name)
# Choose CRDS cache location
use_local_crds_cache = False
crds_cache_tempdir = False
crds_cache_notebook_dir = True
crds_cache_home = False
crds_cache_custom_dir = False
crds_cache_dir_name = ""
if use_local_crds_cache:
if crds_cache_tempdir:
os.environ['CRDS_PATH'] = os.path.join(os.getcwd(), "crds")
elif crds_cache_notebook_dir:
try:
os.environ['CRDS_PATH'] = os.path.join(orig_dir, "crds")
except Exception as e:
os.environ['CRDS_PATH'] = os.path.join(os.getcwd(), "crds")
elif crds_cache_home:
os.environ['CRDS_PATH'] = os.path.join(os.environ['HOME'], 'crds', 'cache')
elif crds_cache_custom_dir:
os.environ['CRDS_PATH'] = crds_cache_dir_name
The library imports relevant to this notebook are aready taken care of by importing the NIRSpec Pipeline Testing Tool.
NOTE: This notebook assumes that the pipeline version to be tested is already installed and its environment is activated.
To be able to run this notebook you need to install NPTT (https://github.com/spacetelescope/nirspec_pipe_testing_tool).
If the installation is successful, you will be able to import NPTT.
import warnings
import psutil
from astropy.io import fits
# Only print a DeprecationWarning the first time it shows up, not every time.
with warnings.catch_warnings():
warnings.simplefilter("once", category=DeprecationWarning)
import jwst
from jwst.pipeline.calwebb_detector1 import Detector1Pipeline
from jwst.assign_wcs.assign_wcs_step import AssignWcsStep
from jwst.msaflagopen.msaflagopen_step import MSAFlagOpenStep
from jwst import datamodels
# The latest version of NPTT is installed in the requirements text file at:
# /jwst_validation_notebooks/environment.yml
# import NPTT
import nirspec_pipe_testing_tool as nptt
# To get data from Artifactory
from ci_watson.artifactory_helpers import get_bigdata
# Make sure that the version used is the right one
pipeline_version = jwst.__version__
nptt_version = nptt.__version__
print("Using jwst pipeline version: ", pipeline_version)
print("Using NPTT version: ", nptt_version)
Using jwst pipeline version: 1.2.3 Using NPTT version: 1.1.17
The test is a comparison of the result of our implementation of the MSA Failed Open Flagging step algorithm versus the pipeline's implementation.
The overlap between the pixels flagged as being affected by failed open shutters are compared in regions large enough in the spectral direction to account for the non-repeatable motion of the grating wheel (a few pixels).
For the test to be considered PASSED, the overlap between the results of the two methods should be greater than or equal to msa_flagging_threshold percent for all failed open shutters affecting more than 100 pixels.
The code for these Multi Object Spectroscopy (MOS) and Integral Field Unit (IFU) tests can be obtained from: https://github.com/spacetelescope/nirspec_pipe_testing_tool/blob/master/nirspec_pipe_testing_tool/calwebb_spec2_pytests/auxiliary_code/msa_flagging_testing.py. This pipeline step and the associated pytests are skipped if data is Fixed Slits (FS) or Bright Object Time Series (BOTS).
The input file is defined in the variable input_file (see section Testing Data Set and Variable Setup).
Step description: https://jwst-pipeline.readthedocs.io/en/latest/jwst/msaflagopen/index.html
Pipeline code: https://github.com/spacetelescope/jwst/tree/master/jwst/msaflagopen
If the test PASSED this means that all slitlets or slices individually passed the test. However, if ony one individual slitlet (for MOS data) or slice (for IFU data) test failed, the whole test will be reported as FAILED.
A short description and link to the page: https://outerspace.stsci.edu/display/JWSTCC/Vanilla+MSA+Failed+Open+Flagging
Acronymns used in this notebook:
pipeline: calibration pipeline
cal_detector1: calibration pipeline Stage 1, detector processing
spec2: calibration pipeline spectroscopic Stage 2, spectroscopic processing
NPTT: NIRSpec Pipeline Testing Tool
The pipeline can be run from the command line in two variants: full or per step.
To run the spec2 pipeline in full use the command:
$ strun jwst.pipeline.Spec2Pipeline jwtest_rate.fits
where jwtest_rate.fits is the output of cal_detector1.
To only run the msa_flagging step, use the command:
$ strun jwst.msaflagopen.msaflagopen_step jwtest_assign_wcs.fits
where jwtest_assign_wcs.fits is the output of the previous step, assign_wcs.
NIRSpec TA data will be run through the calwebb_detector1 and the imaging2 pipelines. The imaging pipeline can be run with the following command:
$ strun jwst.pipeline.Image2Pipeline jwtest_rate.fits
These options are also callable from a script with the testing environment active. The Python call for running the pipeline in full or by step are:
$\gt$ from jwst.pipeline.calwebb_spec2 import Spec2Pipeline
$\gt$ Spec2Pipeline.call(jwtest_rate.fits)
or
$\gt$ from jwst.msaflagopen.msaflagopen_step import msaflagopen_step
$\gt$ msaflagopen_step.call(jwtest_rate.fits)
For the imaging pipeline the call would be as follows:
$\gt$ from jwst.pipeline.calwebb_image2 import Image2Pipeline
$\gt$ Image2Pipeline.call(jwtest_rate.fits)
NPTT can run the spec2 pipeline either in full or per step, as well as the imaging pipeline in full. In this notebook we will use NPTT to run the pipeline and the validation tests. To run NPTT, follow the directions in the corresponding repo page.
All testing data is from the CV3 campaign. We chose these files because this is our most complete data set, i.e. all modes and filter-grating combinations.
Data used for testing:
testing_data = {
'mos_prism_clear':{
'uncal_file_nrs1': 'mos_prism_nrs1_uncal.fits',
'uncal_file_nrs2': 'mos_prism_nrs2_uncal.fits',
'msa_shutter_config': 'V0030006000104_msa.fits' },
'mos_g140m_f100lp':{
'uncal_file_nrs1': 'mos_g140m_line1_NRS1_uncal.fits',
'uncal_file_nrs2': 'mos_g140m_line1_NRS2_uncal.fits',
'msa_shutter_config': 'V8460001000101_msa.fits' },
'ifu_prism_clear':{
'uncal_file_nrs1': 'ifu_prism_nrs1_uncal.fits',
'uncal_file_nrs2': 'ifu_prism_nrs2_uncal.fits',
'msa_shutter_config': None },
'ifu_g395h_f290lp':{
'uncal_file_nrs1': 'ifu_g395h_f290lp_nrs1_uncal.fits',
'uncal_file_nrs2': 'ifu_g395h_f290lp_nrs2_uncal.fits',
'msa_shutter_config': None }
}
# define function to pull data from Artifactory
def get_artifactory_file(data_set_dict, detector):
"""This function creates a list with all the files needed per detector to run the test.
Args:
data_set_dict: dictionary, contains inputs for a specific mode and configuration
detector: string, either nrs1 or nrs2
Returns:
data: list, contains all files needed to run test
"""
files2obtain = ['uncal_file_nrs1', 'msa_shutter_config']
data = []
for file in files2obtain:
data_file = None
try:
if '_nrs' in file and '2' in detector:
file = file.replace('_nrs1', '_nrs2')
data_file = get_bigdata('jwst_validation_notebooks',
'validation_data',
'nirspec_data',
data_set_dict[file])
except TypeError:
data.append(None)
continue
data.append(data_file)
return data
# Set common NPTT switches for this test
# accepted threshold difference with respect to benchmark files
msa_flagging_threshold = 99.5
# other NPTT variables
stellarity = None
operability_ref = None
source_type = None
save_figs = False
show_figs = True
debug = False
verbose = False
# Get the data
results_dict = {}
detectors = ['nrs1', 'nrs2']
for mode_config, data_set_dict in testing_data.items():
for det in detectors:
print('Testing files for detector: ', det)
data = get_artifactory_file(data_set_dict, det)
uncal_file, msa_shutter_config = data
print('Working with uncal_file: ', uncal_file)
uncal_basename = os.path.basename(uncal_file)
# Run the stage 1 pipeline
rate_object = Detector1Pipeline.call(uncal_file)
# Make sure the MSA shutter configuration file is set up correctly
if msa_shutter_config is not None:
msa_metadata = rate_object.meta.instrument.msa_metadata_file
print(msa_metadata)
if msa_metadata is None or msa_metadata == 'N/A':
rate_object.meta.instrument.msa_metadata_file = msa_shutter_config
# Run the stage 2 pipeline steps
try:
pipe_object = AssignWcsStep.call(rate_object)
skip_file = False
except:
print("An error occured with AssignWcs. Likely: No open slits fall on detector", det)
print("Skipping test for this file. \n")
skip_file = True
if not skip_file:
msa_flagging_object = MSAFlagOpenStep.call(pipe_object)
# Run the validation test
%matplotlib inline
result, result_msg, log_msgs = nptt.calwebb_spec2_pytests.auxiliary_code.msa_flagging_testing.run_msa_flagging_testing(
msa_flagging_object,
msa_flagging_threshold = msa_flagging_threshold,
rate_obj = rate_object,
stellarity = stellarity,
operability_ref = operability_ref,
save_figs = save_figs,
show_figs = show_figs,
source_type = source_type,
debug = debug)
else:
result = 'skipped'
# Did the test pass
print("Did msa_flagging validation test pass? ", result, "\n\n")
rd = {uncal_basename: result}
results_dict.update(rd)
# close all open files
psutil.Process().open_files()
closing_files = []
for fd in psutil.Process().open_files():
if data_dir.name in fd.path:
closing_files.append(fd)
for fd in closing_files:
try:
if verbose:
print('Closing file: ', fd)
open(fd.fd).close()
except:
if verbose:
print('File already closed: ', fd)
Testing files for detector: nrs1 Working with uncal_file: /data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/mos_prism_nrs1_uncal.fits
2021-08-27 22:47:54,548 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /grp/crds/cache/references/jwst/jwst_nirspec_pars-detector1pipeline_0004.asdf
2021-08-27 22:47:54,570 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2021-08-27 22:47:54,573 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2021-08-27 22:47:54,575 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2021-08-27 22:47:54,576 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2021-08-27 22:47:54,577 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2021-08-27 22:47:54,578 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2021-08-27 22:47:54,579 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2021-08-27 22:47:54,580 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2021-08-27 22:47:54,581 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2021-08-27 22:47:54,582 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2021-08-27 22:47:54,583 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2021-08-27 22:47:54,584 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2021-08-27 22:47:54,585 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2021-08-27 22:47:54,586 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2021-08-27 22:47:54,587 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2021-08-27 22:47:54,588 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2021-08-27 22:47:54,590 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
2021-08-27 22:47:54,728 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/mos_prism_nrs1_uncal.fits',).
2021-08-27 22:47:54,737 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_calibrated_ramp': False, 'steps': {'group_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dq_init': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'saturation': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'ipc': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'superbias': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'refpix': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}, 'rscd': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'type': 'baseline'}, 'firstframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'lastframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'linearity': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dark_current': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'dark_output': None}, 'reset': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'persistence': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'input_trapsfilled': '', 'flag_pers_cutoff': 40.0, 'save_persistence': False, 'save_trapsfilled': True}, 'jump': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}, 'ramp_fit': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}, 'gain_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}}}
2021-08-27 22:47:54,860 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'mos_prism_nrs1_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'persat', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias', 'trapdensity', 'trappars']
2021-08-27 22:47:54,870 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_dark_0086.fits'.
2021-08-27 22:47:54,872 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_gain_0019.fits'.
2021-08-27 22:47:54,873 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_linearity_0018.fits'.
2021-08-27 22:47:54,875 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_mask_0010.fits'.
2021-08-27 22:47:54,876 - stpipe.Detector1Pipeline - INFO - Prefetch for PERSAT reference file is 'N/A'.
2021-08-27 22:47:54,877 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0018.fits'.
2021-08-27 22:47:54,878 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_refpix_0022.fits'.
2021-08-27 22:47:54,879 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2021-08-27 22:47:54,880 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2021-08-27 22:47:54,880 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_saturation_0020.fits'.
2021-08-27 22:47:54,881 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_superbias_0113.fits'.
2021-08-27 22:47:54,883 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPDENSITY reference file is 'N/A'.
2021-08-27 22:47:54,883 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPPARS reference file is 'N/A'.
2021-08-27 22:47:54,883 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2021-08-27 22:47:55,198 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs1_uncal.fits>,).
2021-08-27 22:47:55,200 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:47:55,323 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2021-08-27 22:47:55,324 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2021-08-27 22:47:55,326 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2021-08-27 22:47:55,444 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs1_uncal.fits>,).
2021-08-27 22:47:55,446 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:47:55,465 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /grp/crds/cache/references/jwst/jwst_nirspec_mask_0010.fits
2021-08-27 22:47:55,794 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2021-08-27 22:47:55,915 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs1_uncal.fits>,).
2021-08-27 22:47:55,917 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:47:55,941 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /grp/crds/cache/references/jwst/jwst_nirspec_saturation_0020.fits
2021-08-27 22:47:56,668 - stpipe.Detector1Pipeline.saturation - INFO - Detected 9869 saturated pixels
2021-08-27 22:47:56,716 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2021-08-27 22:47:56,753 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2021-08-27 22:47:56,854 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs1_uncal.fits>,).
2021-08-27 22:47:56,855 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:47:56,856 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2021-08-27 22:47:56,857 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc done
2021-08-27 22:47:56,949 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs1_uncal.fits>,).
2021-08-27 22:47:56,950 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:47:56,969 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /grp/crds/cache/references/jwst/jwst_nirspec_superbias_0113.fits
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/stdatamodels/validate.py:34: ValidationWarning: While validating meta.exposure.readpatt the following error occurred:
'ALLIRS2' is not one of ['ACQ1', 'ACQ2', 'BRIGHT1', 'BRIGHT2', 'DEEP2', 'DEEP8', 'FAST', 'FASTGRPAVG', 'FASTGRPAVG8', 'FASTGRPAVG16', 'FASTGRPAVG32', 'FASTGRPAVG64', 'FASTR1', 'FASTR100', 'FGS', 'FGS60', 'FGS8370', 'FGS840', 'FGSRAPID', 'FINEGUIDE', 'ID', 'MEDIUM2', 'MEDIUM8', 'NIS', 'NISRAPID', 'NRS', 'NRSIRS2', 'NRSN16R4', 'NRSN32R8', 'NRSN8R2', 'NRSRAPID', 'NRSIRS2RAPID', 'NRSRAPIDD1', 'NRSRAPIDD2', 'NRSRAPIDD6', 'NRSSLOW', 'RAPID', 'SHALLOW2', 'SHALLOW4', 'SLOW', 'SLOWR1', 'TRACK', 'ANY', 'N/A']
Failed validating 'enum' in schema:
OrderedDict([('title', 'Readout pattern'),
('type', 'string'),
('enum',
['ACQ1',
'ACQ2',
'BRIGHT1',
'BRIGHT2',
'DEEP2',
'DEEP8',
'FAST',
'FASTGRPAVG',
'FASTGRPAVG8',
'FASTGRPAVG16',
'FASTGRPAVG32',
'FASTGRPAVG64',
'FASTR1',
'FASTR100',
'FGS',
'FGS60',
'FGS8370',
'FGS840',
'FGSRAPID',
'FINEGUIDE',
'ID',
'MEDIUM2',
'MEDIUM8',
'NIS',
'NISRAPID',
'NRS',
'NRSIRS2',
'NRSN16R4',
'NRSN32R8',
'NRSN8R2',
'NRSRAPID',
'NRSIRS2RAPID',
'NRSRAPIDD1',
'NRSRAPIDD2',
'NRSRAPIDD6',
'NRSSLOW',
'RAPID',
'SHALLOW2',
'SHALLOW4',
'SLOW',
'SLOWR1',
'TRACK',
'ANY',
'N/A']),
('fits_ke ...
warnings.warn(errmsg, ValidationWarning)
2021-08-27 22:48:05,917 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2021-08-27 22:48:06,013 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs1_uncal.fits>,).
2021-08-27 22:48:06,015 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}
2021-08-27 22:48:06,034 - stpipe.Detector1Pipeline.refpix - INFO - Using refpix reference file: /grp/crds/cache/references/jwst/jwst_nirspec_refpix_0022.fits
2021-08-27 22:48:22,917 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2021-08-27 22:48:23,156 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs1_uncal.fits>,).
2021-08-27 22:48:23,158 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:48:23,179 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /grp/crds/cache/references/jwst/jwst_nirspec_linearity_0018.fits
2021-08-27 22:48:23,864 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2021-08-27 22:48:23,976 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs1_uncal.fits>,).
2021-08-27 22:48:23,977 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'dark_output': None}
2021-08-27 22:48:24,034 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /grp/crds/cache/references/jwst/jwst_nirspec_dark_0086.fits
2021-08-27 22:49:35,113 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=4, nframes=1, groupgap=0
2021-08-27 22:49:35,115 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=65, nframes=1, groupgap=0
2021-08-27 22:49:35,687 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2021-08-27 22:49:36,178 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs1_uncal.fits>,).
2021-08-27 22:49:36,179 - stpipe.Detector1Pipeline.jump - INFO - Step jump parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}
2021-08-27 22:49:36,210 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2021-08-27 22:49:36,249 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0019.fits
2021-08-27 22:49:36,324 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0018.fits
2021-08-27 22:49:36,370 - stpipe.Detector1Pipeline.jump - INFO - Using 1 core for jump detection
2021-08-27 22:49:36,569 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2021-08-27 22:49:36,691 - stpipe.Detector1Pipeline.jump - INFO - Working on integration 1:
2021-08-27 22:49:38,359 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 0 pixels with at least one CR and at least four groups
2021-08-27 22:49:38,360 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 19712 pixels with at least one CR and three groups
2021-08-27 22:49:38,361 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 8617 pixels with at least one CR and two groups
2021-08-27 22:49:39,044 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 2.47503 sec
2021-08-27 22:49:39,047 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 2.837023
2021-08-27 22:49:39,050 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2021-08-27 22:49:39,149 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs1_uncal.fits>,).
2021-08-27 22:49:39,150 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}
2021-08-27 22:49:39,215 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0018.fits
2021-08-27 22:49:39,257 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0019.fits
2021-08-27 22:49:39,295 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = ols
2021-08-27 22:49:39,295 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/jwst/ramp_fitting/ols_fit.py:1169: RuntimeWarning: invalid value encountered in multiply
var_p4[num_int, :, :, :] *= (segs_4[num_int, :, :, :] > 0)
2021-08-27 22:49:52,154 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of groups per integration: 4
2021-08-27 22:49:52,154 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of integrations: 1
2021-08-27 22:49:52,357 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2021-08-27 22:49:52,486 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from mos_prism_nrs1_uncal.fits>,).
2021-08-27 22:49:52,488 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scale', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:49:52,541 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2021-08-27 22:49:52,542 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2021-08-27 22:49:52,544 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 22:49:52,630 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from mos_prism_nrs1_uncal.fits>,).
2021-08-27 22:49:52,631 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scaleints', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:49:52,684 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2021-08-27 22:49:52,685 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2021-08-27 22:49:52,687 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 22:49:52,688 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2021-08-27 22:49:52,689 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2021-08-27 22:49:52,698 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 22:49:52,790 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<ImageModel(2048, 2048) from mos_prism_nrs1_uncal.fits>,).
2021-08-27 22:49:52,791 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.25, 'sip_degree': None, 'sip_max_inv_pix_error': 0.25, 'sip_inv_degree': None, 'sip_npoints': 32, 'slit_y_low': -0.55, 'slit_y_high': 0.55}
N/A
2021-08-27 22:49:52,924 - stpipe.AssignWcsStep - INFO - Retrieving open MSA slitlets for msa_metadata_id = 1 and dither_index = 1
2021-08-27 22:49:52,957 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.0332140289247036 deg
2021-08-27 22:49:52,958 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3367482721805573 deg
2021-08-27 22:49:52,958 - stpipe.AssignWcsStep - INFO - theta_y correction: -1.0907214098475986e-05 deg
2021-08-27 22:49:52,959 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:49:53,193 - stpipe.AssignWcsStep - INFO - Slits projected on detector NRS1: [3, 4]
2021-08-27 22:49:53,194 - stpipe.AssignWcsStep - INFO - Computing WCS for 2 open slitlets
2021-08-27 22:49:53,220 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.0332140289247036 deg
2021-08-27 22:49:53,221 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3367482721805573 deg
2021-08-27 22:49:53,221 - stpipe.AssignWcsStep - INFO - theta_y correction: -1.0907214098475986e-05 deg
2021-08-27 22:49:53,222 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:49:53,231 - stpipe.AssignWcsStep - INFO - SPORDER= 0, wrange=[6e-07, 5.3e-06]
2021-08-27 22:49:53,344 - stpipe.AssignWcsStep - INFO - There are 0 open slits in quadrant 1
2021-08-27 22:49:53,345 - stpipe.AssignWcsStep - INFO - There are 0 open slits in quadrant 2
2021-08-27 22:49:53,345 - stpipe.AssignWcsStep - INFO - There are 1 open slits in quadrant 3
2021-08-27 22:49:53,354 - stpipe.AssignWcsStep - INFO - There are 1 open slits in quadrant 4
2021-08-27 22:49:53,363 - stpipe.AssignWcsStep - INFO - There are 0 open slits in quadrant 5
2021-08-27 22:49:53,483 - stpipe.AssignWcsStep - INFO - Created a NIRSPEC nrs_msaspec pipeline with references {'distortion': None, 'filteroffset': None, 'specwcs': None, 'regions': None, 'wavelengthrange': '/grp/crds/cache/references/jwst/jwst_nirspec_wavelengthrange_0004.asdf', 'camera': '/grp/crds/cache/references/jwst/jwst_nirspec_camera_0004.asdf', 'collimator': '/grp/crds/cache/references/jwst/jwst_nirspec_collimator_0004.asdf', 'disperser': '/grp/crds/cache/references/jwst/jwst_nirspec_disperser_0034.asdf', 'fore': '/grp/crds/cache/references/jwst/jwst_nirspec_fore_0028.asdf', 'fpa': '/grp/crds/cache/references/jwst/jwst_nirspec_fpa_0005.asdf', 'msa': '/grp/crds/cache/references/jwst/jwst_nirspec_msa_0005.asdf', 'ote': '/grp/crds/cache/references/jwst/jwst_nirspec_ote_0005.asdf', 'ifupost': None, 'ifufore': None, 'ifuslicer': None, 'msametafile': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/V0030006000104_msa.fits'}
2021-08-27 22:49:53,624 - stpipe.AssignWcsStep - INFO - COMPLETED assign_wcs
2021-08-27 22:49:53,630 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep done
2021-08-27 22:49:53,638 - stpipe.MSAFlagOpenStep - INFO - MSAFlagOpenStep instance created.
2021-08-27 22:49:53,746 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep running with args (<ImageModel(2048, 2048) from mos_prism_nrs1_uncal.fits>,).
2021-08-27 22:49:53,747 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}
2021-08-27 22:49:53,763 - stpipe.MSAFlagOpenStep - INFO - Using reference file /grp/crds/cache/references/jwst/jwst_nirspec_msaoper_0001.json
2021-08-27 22:49:53,764 - stpipe.JwstStep - INFO - JwstStep instance created.
2021-08-27 22:49:53,883 - stpipe.MSAFlagOpenStep - INFO - gwa_ytilt is 0.0332140289247036 deg
2021-08-27 22:49:53,884 - stpipe.MSAFlagOpenStep - INFO - gwa_xtilt is 0.3367482721805573 deg
2021-08-27 22:49:53,884 - stpipe.MSAFlagOpenStep - INFO - theta_y correction: -1.0907214098475986e-05 deg
2021-08-27 22:49:53,885 - stpipe.MSAFlagOpenStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:49:53,894 - stpipe.MSAFlagOpenStep - INFO - SPORDER= 0, wrange=[6e-07, 5.3e-06]
2021-08-27 22:49:54,010 - stpipe.MSAFlagOpenStep - INFO - There are 5 open slits in quadrant 1
2021-08-27 22:49:54,051 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 2
2021-08-27 22:49:54,076 - stpipe.MSAFlagOpenStep - INFO - There are 9 open slits in quadrant 3
2021-08-27 22:49:54,150 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 4
2021-08-27 22:49:54,176 - stpipe.MSAFlagOpenStep - INFO - There are 0 open slits in quadrant 5
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/gwcs/geometry.py:203: RuntimeWarning: invalid value encountered in remainder
lon = np.mod(lon, 360.0 * u.deg if nquant else 360.0)
2021-08-27 22:50:05,017 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep done
2021-08-27 22:50:06,353 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 22:50:06,362 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 22:50:06,563 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<ImageModel(2048, 2048) from mos_prism_nrs1_uncal.fits>,).
2021-08-27 22:50:06,565 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.25, 'sip_degree': None, 'sip_max_inv_pix_error': 0.25, 'sip_inv_degree': None, 'sip_npoints': 32, 'slit_y_low': -0.45, 'slit_y_high': 0.45}
2021-08-27 22:50:06,695 - stpipe.AssignWcsStep - INFO - Retrieving open MSA slitlets for msa_metadata_id = 1 and dither_index = 1
2021-08-27 22:50:06,742 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.0332140289247036 deg
2021-08-27 22:50:06,742 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3367482721805573 deg
2021-08-27 22:50:06,743 - stpipe.AssignWcsStep - INFO - theta_y correction: -1.0907214098475986e-05 deg
2021-08-27 22:50:06,744 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:50:06,911 - stpipe.AssignWcsStep - INFO - Removing slit 1 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:50:06,947 - stpipe.AssignWcsStep - INFO - Removing slit 2 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:50:07,087 - stpipe.AssignWcsStep - INFO - Removing slit 6 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:50:07,124 - stpipe.AssignWcsStep - INFO - Removing slit 7 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:50:07,159 - stpipe.AssignWcsStep - INFO - Removing slit 8 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:50:07,546 - stpipe.AssignWcsStep - INFO - Slits projected on detector NRS1: [3, 4, 5, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
2021-08-27 22:50:07,547 - stpipe.AssignWcsStep - INFO - Computing WCS for 14 open slitlets
2021-08-27 22:50:07,577 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.0332140289247036 deg
2021-08-27 22:50:07,588 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3367482721805573 deg
2021-08-27 22:50:07,588 - stpipe.AssignWcsStep - INFO - theta_y correction: -1.0907214098475986e-05 deg
2021-08-27 22:50:07,590 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:50:07,600 - stpipe.AssignWcsStep - INFO - SPORDER= 0, wrange=[6e-07, 5.3e-06]
2021-08-27 22:50:07,714 - stpipe.AssignWcsStep - INFO - There are 3 open slits in quadrant 1
2021-08-27 22:50:07,739 - stpipe.AssignWcsStep - INFO - There are 0 open slits in quadrant 2
2021-08-27 22:50:07,739 - stpipe.AssignWcsStep - INFO - There are 9 open slits in quadrant 3
2021-08-27 22:50:07,808 - stpipe.AssignWcsStep - INFO - There are 2 open slits in quadrant 4
2021-08-27 22:50:07,823 - stpipe.AssignWcsStep - INFO - There are 0 open slits in quadrant 5
2021-08-27 22:50:08,074 - stpipe.AssignWcsStep - INFO - Created a NIRSPEC nrs_msaspec pipeline with references {'distortion': None, 'filteroffset': None, 'specwcs': None, 'regions': None, 'wavelengthrange': '/grp/crds/cache/references/jwst/jwst_nirspec_wavelengthrange_0004.asdf', 'camera': '/grp/crds/cache/references/jwst/jwst_nirspec_camera_0004.asdf', 'collimator': '/grp/crds/cache/references/jwst/jwst_nirspec_collimator_0004.asdf', 'disperser': '/grp/crds/cache/references/jwst/jwst_nirspec_disperser_0034.asdf', 'fore': '/grp/crds/cache/references/jwst/jwst_nirspec_fore_0028.asdf', 'fpa': '/grp/crds/cache/references/jwst/jwst_nirspec_fpa_0005.asdf', 'msa': '/grp/crds/cache/references/jwst/jwst_nirspec_msa_0005.asdf', 'ote': '/grp/crds/cache/references/jwst/jwst_nirspec_ote_0005.asdf', 'ifupost': None, 'ifufore': None, 'ifuslicer': None, 'msametafile': 'fopens_metafile_msa.fits'}
2021-08-27 22:50:08,362 - stpipe.AssignWcsStep - INFO - COMPLETED assign_wcs
2021-08-27 22:50:08,368 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep done
Working with slit/slice: 3 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[2007 2048] [1629 1641]] Max value in slity array (ignoring NANs): 0.49934753059464704 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 4 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[2017 2048] [1319 1331]] Max value in slity array (ignoring NANs): 0.4992600265017523 percentage of F/O trace that was flagged: 96.62162162162163
Working with slit/slice: 5 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[1873 2048] [1590 1601]] Max value in slity array (ignoring NANs): 0.49970473402574966 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 9 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[1511 1939] [1261 1271]] Max value in slity array (ignoring NANs): 0.47541638939121267 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 10 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[1515 1943] [1230 1241]] Max value in slity array (ignoring NANs): 0.4999677623917731 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 11 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[1160 1594] [1827 1837]] Max value in slity array (ignoring NANs): 0.4975789418957247 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 12 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[1213 1643] [1251 1261]] Max value in slity array (ignoring NANs): 0.4911821622434934 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 13 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 920 1356] [1692 1703]] Max value in slity array (ignoring NANs): 0.4997273932113973 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 14 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 819 1255] [1632 1643]] Max value in slity array (ignoring NANs): 0.4998008389664905 percentage of F/O trace that was flagged: 99.785061794734
Working with slit/slice: 15 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 736 1172] [1376 1388]] Max value in slity array (ignoring NANs): 0.49973439389336194 percentage of F/O trace that was flagged: 99.7874601487779
Working with slit/slice: 16 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 656 1096] [1862 1874]] Max value in slity array (ignoring NANs): 0.49919217087642653 percentage of F/O trace that was flagged: 99.7872340425532
Working with slit/slice: 17 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 707 1144] [1517 1528]] Max value in slity array (ignoring NANs): 0.49950881420114257 percentage of F/O trace that was flagged: 99.78401727861771
Working with slit/slice: 18 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 859 1287] [ 205 215]] Max value in slity array (ignoring NANs): 0.4985413500738489 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 19 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 761 1193] [ 623 634]] Max value in slity array (ignoring NANs): 0.4996526774597415 percentage of F/O trace that was flagged: 99.78563772775992
*** Final result for msa_flagging test will be reported as FAILED ***
('* MSA flagging validation test took ', '15.477138757705688 seconds to finish.')
Did msa_flagging validation test pass? False
Testing files for detector: nrs2
Working with uncal_file: /data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/mos_prism_nrs2_uncal.fits
2021-08-27 22:50:21,308 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /grp/crds/cache/references/jwst/jwst_nirspec_pars-detector1pipeline_0004.asdf
2021-08-27 22:50:21,323 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2021-08-27 22:50:21,324 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2021-08-27 22:50:21,326 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2021-08-27 22:50:21,327 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2021-08-27 22:50:21,328 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2021-08-27 22:50:21,329 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2021-08-27 22:50:21,330 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2021-08-27 22:50:21,331 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2021-08-27 22:50:21,332 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2021-08-27 22:50:21,333 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2021-08-27 22:50:21,335 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2021-08-27 22:50:21,336 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2021-08-27 22:50:21,337 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2021-08-27 22:50:21,338 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2021-08-27 22:50:21,340 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2021-08-27 22:50:21,341 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2021-08-27 22:50:21,342 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/stpipe/step.py:359: ResourceWarning: unclosed file <_io.FileIO name='/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/mos_prism_nrs2_uncal.fits' mode='rb' closefd=True>
gc.collect()
2021-08-27 22:50:21,635 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/mos_prism_nrs2_uncal.fits',).
2021-08-27 22:50:21,645 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_calibrated_ramp': False, 'steps': {'group_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dq_init': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'saturation': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'ipc': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'superbias': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'refpix': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}, 'rscd': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'type': 'baseline'}, 'firstframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'lastframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'linearity': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dark_current': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'dark_output': None}, 'reset': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'persistence': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'input_trapsfilled': '', 'flag_pers_cutoff': 40.0, 'save_persistence': False, 'save_trapsfilled': True}, 'jump': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}, 'ramp_fit': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}, 'gain_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}}}
2021-08-27 22:50:21,762 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'mos_prism_nrs2_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'persat', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias', 'trapdensity', 'trappars']
2021-08-27 22:50:21,773 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_dark_0087.fits'.
2021-08-27 22:50:21,775 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_gain_0018.fits'.
2021-08-27 22:50:21,776 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_linearity_0021.fits'.
2021-08-27 22:50:21,777 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_mask_0011.fits'.
2021-08-27 22:50:21,779 - stpipe.Detector1Pipeline - INFO - Prefetch for PERSAT reference file is 'N/A'.
2021-08-27 22:50:21,779 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0017.fits'.
2021-08-27 22:50:21,781 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_refpix_0020.fits'.
2021-08-27 22:50:21,783 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2021-08-27 22:50:21,783 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2021-08-27 22:50:21,784 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_saturation_0021.fits'.
2021-08-27 22:50:21,785 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_superbias_0114.fits'.
2021-08-27 22:50:21,787 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPDENSITY reference file is 'N/A'.
2021-08-27 22:50:21,787 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPPARS reference file is 'N/A'.
2021-08-27 22:50:21,788 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2021-08-27 22:50:22,171 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs2_uncal.fits>,).
2021-08-27 22:50:22,173 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:50:22,315 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2021-08-27 22:50:22,316 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2021-08-27 22:50:22,318 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2021-08-27 22:50:22,505 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs2_uncal.fits>,).
2021-08-27 22:50:22,506 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:50:22,526 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /grp/crds/cache/references/jwst/jwst_nirspec_mask_0011.fits
2021-08-27 22:50:22,878 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2021-08-27 22:50:23,070 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs2_uncal.fits>,).
2021-08-27 22:50:23,072 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:50:23,092 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /grp/crds/cache/references/jwst/jwst_nirspec_saturation_0021.fits
2021-08-27 22:50:23,605 - stpipe.Detector1Pipeline.saturation - INFO - Detected 5379 saturated pixels
2021-08-27 22:50:23,630 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2021-08-27 22:50:23,658 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2021-08-27 22:50:23,850 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs2_uncal.fits>,).
2021-08-27 22:50:23,852 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:50:23,853 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2021-08-27 22:50:23,854 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc done
2021-08-27 22:50:24,033 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs2_uncal.fits>,).
2021-08-27 22:50:24,034 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:50:24,054 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /grp/crds/cache/references/jwst/jwst_nirspec_superbias_0114.fits
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/stdatamodels/validate.py:34: ValidationWarning: While validating meta.exposure.readpatt the following error occurred:
'ALLIRS2' is not one of ['ACQ1', 'ACQ2', 'BRIGHT1', 'BRIGHT2', 'DEEP2', 'DEEP8', 'FAST', 'FASTGRPAVG', 'FASTGRPAVG8', 'FASTGRPAVG16', 'FASTGRPAVG32', 'FASTGRPAVG64', 'FASTR1', 'FASTR100', 'FGS', 'FGS60', 'FGS8370', 'FGS840', 'FGSRAPID', 'FINEGUIDE', 'ID', 'MEDIUM2', 'MEDIUM8', 'NIS', 'NISRAPID', 'NRS', 'NRSIRS2', 'NRSN16R4', 'NRSN32R8', 'NRSN8R2', 'NRSRAPID', 'NRSIRS2RAPID', 'NRSRAPIDD1', 'NRSRAPIDD2', 'NRSRAPIDD6', 'NRSSLOW', 'RAPID', 'SHALLOW2', 'SHALLOW4', 'SLOW', 'SLOWR1', 'TRACK', 'ANY', 'N/A']
Failed validating 'enum' in schema:
OrderedDict([('title', 'Readout pattern'),
('type', 'string'),
('enum',
['ACQ1',
'ACQ2',
'BRIGHT1',
'BRIGHT2',
'DEEP2',
'DEEP8',
'FAST',
'FASTGRPAVG',
'FASTGRPAVG8',
'FASTGRPAVG16',
'FASTGRPAVG32',
'FASTGRPAVG64',
'FASTR1',
'FASTR100',
'FGS',
'FGS60',
'FGS8370',
'FGS840',
'FGSRAPID',
'FINEGUIDE',
'ID',
'MEDIUM2',
'MEDIUM8',
'NIS',
'NISRAPID',
'NRS',
'NRSIRS2',
'NRSN16R4',
'NRSN32R8',
'NRSN8R2',
'NRSRAPID',
'NRSIRS2RAPID',
'NRSRAPIDD1',
'NRSRAPIDD2',
'NRSRAPIDD6',
'NRSSLOW',
'RAPID',
'SHALLOW2',
'SHALLOW4',
'SLOW',
'SLOWR1',
'TRACK',
'ANY',
'N/A']),
('fits_ke ...
warnings.warn(errmsg, ValidationWarning)
2021-08-27 22:50:33,831 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2021-08-27 22:50:34,024 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs2_uncal.fits>,).
2021-08-27 22:50:34,026 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}
2021-08-27 22:50:34,047 - stpipe.Detector1Pipeline.refpix - INFO - Using refpix reference file: /grp/crds/cache/references/jwst/jwst_nirspec_refpix_0020.fits
2021-08-27 22:50:49,590 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2021-08-27 22:50:49,938 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs2_uncal.fits>,).
2021-08-27 22:50:49,940 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:50:49,964 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /grp/crds/cache/references/jwst/jwst_nirspec_linearity_0021.fits
2021-08-27 22:50:50,610 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2021-08-27 22:50:50,825 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs2_uncal.fits>,).
2021-08-27 22:50:50,827 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'dark_output': None}
2021-08-27 22:50:50,899 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /grp/crds/cache/references/jwst/jwst_nirspec_dark_0087.fits
2021-08-27 22:51:44,431 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=4, nframes=1, groupgap=0
2021-08-27 22:51:44,431 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=65, nframes=1, groupgap=0
2021-08-27 22:51:45,045 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2021-08-27 22:51:45,581 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs2_uncal.fits>,).
2021-08-27 22:51:45,583 - stpipe.Detector1Pipeline.jump - INFO - Step jump parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}
2021-08-27 22:51:45,609 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2021-08-27 22:51:45,639 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0018.fits
2021-08-27 22:51:45,730 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0017.fits
2021-08-27 22:51:45,779 - stpipe.Detector1Pipeline.jump - INFO - Using 1 core for jump detection
2021-08-27 22:51:45,955 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2021-08-27 22:51:46,072 - stpipe.Detector1Pipeline.jump - INFO - Working on integration 1:
2021-08-27 22:51:47,692 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 0 pixels with at least one CR and at least four groups
2021-08-27 22:51:47,693 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 7787 pixels with at least one CR and three groups
2021-08-27 22:51:47,693 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 3636 pixels with at least one CR and two groups
2021-08-27 22:51:48,028 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 2.07197 sec
2021-08-27 22:51:48,030 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 2.421339
2021-08-27 22:51:48,033 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2021-08-27 22:51:48,145 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 4, 3200, 2048) from mos_prism_nrs2_uncal.fits>,).
2021-08-27 22:51:48,147 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}
2021-08-27 22:51:48,214 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0017.fits
2021-08-27 22:51:48,255 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0018.fits
2021-08-27 22:51:48,292 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = ols
2021-08-27 22:51:48,293 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/jwst/ramp_fitting/ols_fit.py:1169: RuntimeWarning: invalid value encountered in multiply
var_p4[num_int, :, :, :] *= (segs_4[num_int, :, :, :] > 0)
2021-08-27 22:52:00,583 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of groups per integration: 4
2021-08-27 22:52:00,584 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of integrations: 1
2021-08-27 22:52:00,791 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2021-08-27 22:52:00,905 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from mos_prism_nrs2_uncal.fits>,).
2021-08-27 22:52:00,907 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scale', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:52:00,965 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2021-08-27 22:52:00,966 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2021-08-27 22:52:00,968 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 22:52:01,070 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from mos_prism_nrs2_uncal.fits>,).
2021-08-27 22:52:01,071 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scaleints', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:52:01,128 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2021-08-27 22:52:01,128 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2021-08-27 22:52:01,131 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 22:52:01,132 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2021-08-27 22:52:01,133 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2021-08-27 22:52:01,142 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 22:52:01,242 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<ImageModel(2048, 2048) from mos_prism_nrs2_uncal.fits>,).
2021-08-27 22:52:01,243 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.25, 'sip_degree': None, 'sip_max_inv_pix_error': 0.25, 'sip_inv_degree': None, 'sip_npoints': 32, 'slit_y_low': -0.55, 'slit_y_high': 0.55}
N/A
2021-08-27 22:52:01,387 - stpipe.AssignWcsStep - INFO - Retrieving open MSA slitlets for msa_metadata_id = 1 and dither_index = 1 2021-08-27 22:52:01,419 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.0332140289247036 deg 2021-08-27 22:52:01,419 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3367482721805573 deg 2021-08-27 22:52:01,420 - stpipe.AssignWcsStep - INFO - theta_y correction: -1.0907214098475986e-05 deg 2021-08-27 22:52:01,421 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg 2021-08-27 22:52:01,601 - stpipe.AssignWcsStep - INFO - Removing slit 3 from the list of open slits because the WCS bounding_box is completely outside the detector. 2021-08-27 22:52:01,637 - stpipe.AssignWcsStep - INFO - Removing slit 4 from the list of open slits because the WCS bounding_box is completely outside the detector. 2021-08-27 22:52:01,638 - stpipe.AssignWcsStep - INFO - Slits projected on detector NRS2: [] 2021-08-27 22:52:01,638 - stpipe.AssignWcsStep - CRITICAL - No open slits fall on detector NRS2.
An error occured with AssignWcs. Likely: No open slits fall on detector nrs2 Skipping test for this file. Did msa_flagging validation test pass? skipped Testing files for detector: nrs1 Working with uncal_file: /data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/mos_g140m_line1_NRS1_uncal.fits
2021-08-27 22:52:02,418 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /grp/crds/cache/references/jwst/jwst_nirspec_pars-detector1pipeline_0004.asdf
2021-08-27 22:52:02,435 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2021-08-27 22:52:02,436 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2021-08-27 22:52:02,437 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2021-08-27 22:52:02,439 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2021-08-27 22:52:02,440 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2021-08-27 22:52:02,441 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2021-08-27 22:52:02,442 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2021-08-27 22:52:02,443 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2021-08-27 22:52:02,444 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2021-08-27 22:52:02,445 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2021-08-27 22:52:02,446 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2021-08-27 22:52:02,447 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2021-08-27 22:52:02,448 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2021-08-27 22:52:02,450 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2021-08-27 22:52:02,451 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2021-08-27 22:52:02,452 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2021-08-27 22:52:02,453 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/stpipe/step.py:359: ResourceWarning: unclosed file <_io.FileIO name='/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/mos_g140m_line1_NRS1_uncal.fits' mode='rb' closefd=True>
gc.collect()
2021-08-27 22:52:02,639 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/mos_g140m_line1_NRS1_uncal.fits',).
2021-08-27 22:52:02,648 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_calibrated_ramp': False, 'steps': {'group_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dq_init': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'saturation': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'ipc': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'superbias': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'refpix': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}, 'rscd': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'type': 'baseline'}, 'firstframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'lastframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'linearity': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dark_current': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'dark_output': None}, 'reset': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'persistence': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'input_trapsfilled': '', 'flag_pers_cutoff': 40.0, 'save_persistence': False, 'save_trapsfilled': True}, 'jump': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}, 'ramp_fit': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}, 'gain_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}}}
2021-08-27 22:52:02,765 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'mos_g140m_line1_NRS1_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'persat', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias', 'trapdensity', 'trappars']
2021-08-27 22:52:02,773 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_dark_0140.fits'.
2021-08-27 22:52:02,774 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_gain_0019.fits'.
2021-08-27 22:52:02,776 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_linearity_0019.fits'.
2021-08-27 22:52:02,777 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_mask_0024.fits'.
2021-08-27 22:52:02,779 - stpipe.Detector1Pipeline - INFO - Prefetch for PERSAT reference file is 'N/A'.
2021-08-27 22:52:02,779 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0019.fits'.
2021-08-27 22:52:02,781 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2021-08-27 22:52:02,781 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2021-08-27 22:52:02,782 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2021-08-27 22:52:02,782 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_saturation_0023.fits'.
2021-08-27 22:52:02,784 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_superbias_0087.fits'.
2021-08-27 22:52:02,785 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPDENSITY reference file is 'N/A'.
2021-08-27 22:52:02,786 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPPARS reference file is 'N/A'.
2021-08-27 22:52:02,786 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2021-08-27 22:52:03,141 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS1_uncal.fits>,).
2021-08-27 22:52:03,142 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:52:03,243 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2021-08-27 22:52:03,244 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2021-08-27 22:52:03,245 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2021-08-27 22:52:03,435 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS1_uncal.fits>,).
2021-08-27 22:52:03,437 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:52:03,456 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /grp/crds/cache/references/jwst/jwst_nirspec_mask_0024.fits
2021-08-27 22:52:03,832 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2021-08-27 22:52:04,021 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS1_uncal.fits>,).
2021-08-27 22:52:04,023 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:52:04,043 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /grp/crds/cache/references/jwst/jwst_nirspec_saturation_0023.fits
2021-08-27 22:52:05,191 - stpipe.Detector1Pipeline.saturation - INFO - Detected 58922 saturated pixels
2021-08-27 22:52:05,211 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2021-08-27 22:52:05,221 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2021-08-27 22:52:05,338 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS1_uncal.fits>,).
2021-08-27 22:52:05,340 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:52:05,340 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2021-08-27 22:52:05,341 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc done
2021-08-27 22:52:05,438 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS1_uncal.fits>,).
2021-08-27 22:52:05,439 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:52:05,457 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /grp/crds/cache/references/jwst/jwst_nirspec_superbias_0087.fits
2021-08-27 22:52:05,691 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2021-08-27 22:52:05,791 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS1_uncal.fits>,).
2021-08-27 22:52:05,792 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}
2021-08-27 22:52:05,800 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2021-08-27 22:52:05,800 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2021-08-27 22:52:05,801 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2021-08-27 22:52:05,801 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.000000
2021-08-27 22:52:05,801 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = True
2021-08-27 22:52:08,437 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2021-08-27 22:52:08,535 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS1_uncal.fits>,).
2021-08-27 22:52:08,537 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:52:08,555 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /grp/crds/cache/references/jwst/jwst_nirspec_linearity_0019.fits
2021-08-27 22:52:13,881 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2021-08-27 22:52:14,068 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS1_uncal.fits>,).
2021-08-27 22:52:14,069 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'dark_output': None}
2021-08-27 22:52:14,089 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /grp/crds/cache/references/jwst/jwst_nirspec_dark_0140.fits
2021-08-27 22:52:14,299 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=6, nframes=1, groupgap=0
2021-08-27 22:52:14,300 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=10, nframes=1, groupgap=0
2021-08-27 22:52:14,492 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2021-08-27 22:52:14,685 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS1_uncal.fits>,).
2021-08-27 22:52:14,687 - stpipe.Detector1Pipeline.jump - INFO - Step jump parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}
2021-08-27 22:52:14,696 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2021-08-27 22:52:14,707 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0019.fits
2021-08-27 22:52:14,764 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0019.fits
2021-08-27 22:52:14,812 - stpipe.Detector1Pipeline.jump - INFO - Using 1 core for jump detection
2021-08-27 22:52:15,007 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2021-08-27 22:52:15,177 - stpipe.Detector1Pipeline.jump - INFO - Working on integration 1:
2021-08-27 22:52:16,629 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 39308 pixels with at least one CR and at least four groups
2021-08-27 22:52:16,630 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 3301 pixels with at least one CR and three groups
2021-08-27 22:52:16,631 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 44253 pixels with at least one CR and two groups
2021-08-27 22:52:18,785 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 3.77701 sec
2021-08-27 22:52:18,788 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 4.091675
2021-08-27 22:52:18,791 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2021-08-27 22:52:18,984 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS1_uncal.fits>,).
2021-08-27 22:52:18,985 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}
2021-08-27 22:52:19,015 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0019.fits
2021-08-27 22:52:19,063 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0019.fits
2021-08-27 22:52:19,113 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = ols
2021-08-27 22:52:19,113 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/jwst/ramp_fitting/ols_fit.py:1169: RuntimeWarning: invalid value encountered in multiply
var_p4[num_int, :, :, :] *= (segs_4[num_int, :, :, :] > 0)
2021-08-27 22:52:40,614 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of groups per integration: 6
2021-08-27 22:52:40,615 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of integrations: 1
2021-08-27 22:52:40,771 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2021-08-27 22:52:40,887 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from mos_g140m_line1_NRS1_uncal.fits>,).
2021-08-27 22:52:40,888 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scale', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:52:40,950 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2021-08-27 22:52:40,950 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2021-08-27 22:52:40,952 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 22:52:41,046 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from mos_g140m_line1_NRS1_uncal.fits>,).
2021-08-27 22:52:41,048 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scaleints', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:52:41,102 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2021-08-27 22:52:41,102 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2021-08-27 22:52:41,104 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 22:52:41,105 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2021-08-27 22:52:41,106 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2021-08-27 22:52:41,114 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 22:52:41,209 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<ImageModel(2048, 2048) from mos_g140m_line1_NRS1_uncal.fits>,).
2021-08-27 22:52:41,210 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.25, 'sip_degree': None, 'sip_max_inv_pix_error': 0.25, 'sip_inv_degree': None, 'sip_npoints': 32, 'slit_y_low': -0.55, 'slit_y_high': 0.55}
V8460001000101_msa.fits
2021-08-27 22:52:41,356 - stpipe.AssignWcsStep - INFO - Retrieving open MSA slitlets for msa_metadata_id = 1 and dither_index = 1
2021-08-27 22:52:41,571 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.1260581910610199 deg
2021-08-27 22:52:41,572 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3316612243652344 deg
2021-08-27 22:52:41,572 - stpipe.AssignWcsStep - INFO - theta_y correction: 0.0002496099796717191 deg
2021-08-27 22:52:41,574 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:52:41,818 - stpipe.AssignWcsStep - INFO - Removing slit 6 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:41,830 - stpipe.AssignWcsStep - INFO - Removing slit 7 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:41,841 - stpipe.AssignWcsStep - INFO - Removing slit 8 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:41,853 - stpipe.AssignWcsStep - INFO - Removing slit 9 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:41,866 - stpipe.AssignWcsStep - INFO - Removing slit 10 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:41,877 - stpipe.AssignWcsStep - INFO - Removing slit 11 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:41,889 - stpipe.AssignWcsStep - INFO - Removing slit 14 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:41,901 - stpipe.AssignWcsStep - INFO - Removing slit 15 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:41,912 - stpipe.AssignWcsStep - INFO - Removing slit 16 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:41,924 - stpipe.AssignWcsStep - INFO - Removing slit 17 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:41,936 - stpipe.AssignWcsStep - INFO - Removing slit 18 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:41,948 - stpipe.AssignWcsStep - INFO - Removing slit 19 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:41,959 - stpipe.AssignWcsStep - INFO - Removing slit 21 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:41,971 - stpipe.AssignWcsStep - INFO - Removing slit 25 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:41,982 - stpipe.AssignWcsStep - INFO - Removing slit 26 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:41,994 - stpipe.AssignWcsStep - INFO - Removing slit 27 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,006 - stpipe.AssignWcsStep - INFO - Removing slit 29 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,017 - stpipe.AssignWcsStep - INFO - Removing slit 31 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,029 - stpipe.AssignWcsStep - INFO - Removing slit 32 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,041 - stpipe.AssignWcsStep - INFO - Removing slit 33 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,053 - stpipe.AssignWcsStep - INFO - Removing slit 35 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,066 - stpipe.AssignWcsStep - INFO - Removing slit 37 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,078 - stpipe.AssignWcsStep - INFO - Removing slit 38 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,090 - stpipe.AssignWcsStep - INFO - Removing slit 39 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,101 - stpipe.AssignWcsStep - INFO - Removing slit 41 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,135 - stpipe.AssignWcsStep - INFO - Removing slit 1 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,147 - stpipe.AssignWcsStep - INFO - Removing slit 2 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,159 - stpipe.AssignWcsStep - INFO - Removing slit 3 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,170 - stpipe.AssignWcsStep - INFO - Removing slit 4 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,182 - stpipe.AssignWcsStep - INFO - Removing slit 5 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,193 - stpipe.AssignWcsStep - INFO - Removing slit 12 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,205 - stpipe.AssignWcsStep - INFO - Removing slit 13 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,217 - stpipe.AssignWcsStep - INFO - Removing slit 20 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,228 - stpipe.AssignWcsStep - INFO - Removing slit 22 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,240 - stpipe.AssignWcsStep - INFO - Removing slit 23 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,252 - stpipe.AssignWcsStep - INFO - Removing slit 24 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,265 - stpipe.AssignWcsStep - INFO - Removing slit 28 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,276 - stpipe.AssignWcsStep - INFO - Removing slit 30 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,288 - stpipe.AssignWcsStep - INFO - Removing slit 34 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,300 - stpipe.AssignWcsStep - INFO - Removing slit 36 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,312 - stpipe.AssignWcsStep - INFO - Removing slit 40 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:42,590 - stpipe.AssignWcsStep - INFO - Slits projected on detector NRS1: [42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68]
2021-08-27 22:52:42,591 - stpipe.AssignWcsStep - INFO - Computing WCS for 27 open slitlets
2021-08-27 22:52:42,616 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.1260581910610199 deg
2021-08-27 22:52:42,616 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3316612243652344 deg
2021-08-27 22:52:42,617 - stpipe.AssignWcsStep - INFO - theta_y correction: 0.0002496099796717191 deg
2021-08-27 22:52:42,618 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:52:42,627 - stpipe.AssignWcsStep - INFO - SPORDER= -1, wrange=[9.7e-07, 1.89e-06]
2021-08-27 22:52:42,736 - stpipe.AssignWcsStep - INFO - There are 2 open slits in quadrant 1
2021-08-27 22:52:42,752 - stpipe.AssignWcsStep - INFO - There are 11 open slits in quadrant 2
2021-08-27 22:52:42,835 - stpipe.AssignWcsStep - INFO - There are 4 open slits in quadrant 3
2021-08-27 22:52:42,989 - stpipe.AssignWcsStep - INFO - There are 10 open slits in quadrant 4
2021-08-27 22:52:43,069 - stpipe.AssignWcsStep - INFO - There are 0 open slits in quadrant 5
2021-08-27 22:52:43,199 - stpipe.AssignWcsStep - INFO - Created a NIRSPEC nrs_msaspec pipeline with references {'distortion': None, 'filteroffset': None, 'specwcs': None, 'regions': None, 'wavelengthrange': '/grp/crds/cache/references/jwst/jwst_nirspec_wavelengthrange_0004.asdf', 'camera': '/grp/crds/cache/references/jwst/jwst_nirspec_camera_0004.asdf', 'collimator': '/grp/crds/cache/references/jwst/jwst_nirspec_collimator_0004.asdf', 'disperser': '/grp/crds/cache/references/jwst/jwst_nirspec_disperser_0032.asdf', 'fore': '/grp/crds/cache/references/jwst/jwst_nirspec_fore_0023.asdf', 'fpa': '/grp/crds/cache/references/jwst/jwst_nirspec_fpa_0005.asdf', 'msa': '/grp/crds/cache/references/jwst/jwst_nirspec_msa_0005.asdf', 'ote': '/grp/crds/cache/references/jwst/jwst_nirspec_ote_0005.asdf', 'ifupost': None, 'ifufore': None, 'ifuslicer': None, 'msametafile': 'V8460001000101_msa.fits'}
2021-08-27 22:52:43,750 - stpipe.AssignWcsStep - INFO - COMPLETED assign_wcs
2021-08-27 22:52:43,756 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep done
2021-08-27 22:52:43,765 - stpipe.MSAFlagOpenStep - INFO - MSAFlagOpenStep instance created.
2021-08-27 22:52:43,888 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep running with args (<ImageModel(2048, 2048) from mos_g140m_line1_NRS1_uncal.fits>,).
2021-08-27 22:52:43,889 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}
2021-08-27 22:52:43,905 - stpipe.MSAFlagOpenStep - INFO - Using reference file /grp/crds/cache/references/jwst/jwst_nirspec_msaoper_0001.json
2021-08-27 22:52:43,906 - stpipe.JwstStep - INFO - JwstStep instance created.
2021-08-27 22:52:44,018 - stpipe.MSAFlagOpenStep - INFO - gwa_ytilt is 0.1260581910610199 deg
2021-08-27 22:52:44,019 - stpipe.MSAFlagOpenStep - INFO - gwa_xtilt is 0.3316612243652344 deg
2021-08-27 22:52:44,019 - stpipe.MSAFlagOpenStep - INFO - theta_y correction: 0.0002496099796717191 deg
2021-08-27 22:52:44,020 - stpipe.MSAFlagOpenStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:52:44,028 - stpipe.MSAFlagOpenStep - INFO - SPORDER= -1, wrange=[9.7e-07, 1.89e-06]
2021-08-27 22:52:44,138 - stpipe.MSAFlagOpenStep - INFO - There are 5 open slits in quadrant 1
2021-08-27 22:52:44,176 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 2
2021-08-27 22:52:44,199 - stpipe.MSAFlagOpenStep - INFO - There are 9 open slits in quadrant 3
2021-08-27 22:52:44,267 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 4
2021-08-27 22:52:44,290 - stpipe.MSAFlagOpenStep - INFO - There are 0 open slits in quadrant 5
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/gwcs/geometry.py:203: RuntimeWarning: invalid value encountered in remainder
lon = np.mod(lon, 360.0 * u.deg if nquant else 360.0)
2021-08-27 22:52:55,430 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep done
2021-08-27 22:52:56,162 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 22:52:56,172 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 22:52:56,440 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<ImageModel(2048, 2048) from mos_g140m_line1_NRS1_uncal.fits>,).
2021-08-27 22:52:56,442 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.25, 'sip_degree': None, 'sip_max_inv_pix_error': 0.25, 'sip_inv_degree': None, 'sip_npoints': 32, 'slit_y_low': -0.45, 'slit_y_high': 0.45}
2021-08-27 22:52:56,577 - stpipe.AssignWcsStep - INFO - Retrieving open MSA slitlets for msa_metadata_id = 1 and dither_index = 1
2021-08-27 22:52:56,627 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.1260581910610199 deg
2021-08-27 22:52:56,628 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3316612243652344 deg
2021-08-27 22:52:56,628 - stpipe.AssignWcsStep - INFO - theta_y correction: 0.0002496099796717191 deg
2021-08-27 22:52:56,629 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:52:56,775 - stpipe.AssignWcsStep - INFO - Removing slit 1 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:56,787 - stpipe.AssignWcsStep - INFO - Removing slit 2 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:56,833 - stpipe.AssignWcsStep - INFO - Removing slit 6 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:52:56,979 - stpipe.AssignWcsStep - INFO - Slits projected on detector NRS1: [3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
2021-08-27 22:52:56,979 - stpipe.AssignWcsStep - INFO - Computing WCS for 16 open slitlets
2021-08-27 22:52:57,006 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.1260581910610199 deg
2021-08-27 22:52:57,007 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3316612243652344 deg
2021-08-27 22:52:57,008 - stpipe.AssignWcsStep - INFO - theta_y correction: 0.0002496099796717191 deg
2021-08-27 22:52:57,008 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:52:57,017 - stpipe.AssignWcsStep - INFO - SPORDER= -1, wrange=[9.7e-07, 1.89e-06]
2021-08-27 22:52:57,129 - stpipe.AssignWcsStep - INFO - There are 3 open slits in quadrant 1
2021-08-27 22:52:57,154 - stpipe.AssignWcsStep - INFO - There are 2 open slits in quadrant 2
2021-08-27 22:52:57,170 - stpipe.AssignWcsStep - INFO - There are 9 open slits in quadrant 3
2021-08-27 22:52:57,241 - stpipe.AssignWcsStep - INFO - There are 2 open slits in quadrant 4
2021-08-27 22:52:57,258 - stpipe.AssignWcsStep - INFO - There are 0 open slits in quadrant 5
2021-08-27 22:52:57,377 - stpipe.AssignWcsStep - INFO - Created a NIRSPEC nrs_msaspec pipeline with references {'distortion': None, 'filteroffset': None, 'specwcs': None, 'regions': None, 'wavelengthrange': '/grp/crds/cache/references/jwst/jwst_nirspec_wavelengthrange_0004.asdf', 'camera': '/grp/crds/cache/references/jwst/jwst_nirspec_camera_0004.asdf', 'collimator': '/grp/crds/cache/references/jwst/jwst_nirspec_collimator_0004.asdf', 'disperser': '/grp/crds/cache/references/jwst/jwst_nirspec_disperser_0032.asdf', 'fore': '/grp/crds/cache/references/jwst/jwst_nirspec_fore_0023.asdf', 'fpa': '/grp/crds/cache/references/jwst/jwst_nirspec_fpa_0005.asdf', 'msa': '/grp/crds/cache/references/jwst/jwst_nirspec_msa_0005.asdf', 'ote': '/grp/crds/cache/references/jwst/jwst_nirspec_ote_0005.asdf', 'ifupost': None, 'ifufore': None, 'ifuslicer': None, 'msametafile': 'fopens_metafile_msa.fits'}
2021-08-27 22:52:57,821 - stpipe.AssignWcsStep - INFO - COMPLETED assign_wcs
2021-08-27 22:52:57,827 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep done
Working with slit/slice: 3 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[1677 2048] [1630 1645]] Max value in slity array (ignoring NANs): 0.49983590702157094 percentage of F/O trace that was flagged: 99.80481457384516
Working with slit/slice: 4 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[1628 2048] [1320 1334]] Max value in slity array (ignoring NANs): 0.4999416262418918 percentage of F/O trace that was flagged: 99.78712080894093
Working with slit/slice: 5 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[1536 2048] [1591 1604]] Max value in slity array (ignoring NANs): 0.499395872247531 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 7 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[1531 2048] [ 239 251]] Max value in slity array (ignoring NANs): 0.49982577845792203 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 8 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[1510 2048] [ 163 175]] Max value in slity array (ignoring NANs): 0.4999363264129128 percentage of F/O trace that was flagged: 99.83193277310924
Working with slit/slice: 9 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[1114 2048] [1260 1273]] Max value in slity array (ignoring NANs): 0.49973662248892214 percentage of F/O trace that was flagged: 99.9002244948865
Working with slit/slice: 10 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[1113 2048] [1230 1243]] Max value in slity array (ignoring NANs): 0.499843929708657 percentage of F/O trace that was flagged: 99.89997499374844
Working with slit/slice: 11 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 881 2048] [1825 1841]] Max value in slity array (ignoring NANs): 0.49948331088759856 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 12 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 822 2048] [1249 1265]] Max value in slity array (ignoring NANs): 0.4997723843713606 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 13 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 622 2048] [1688 1708]] Max value in slity array (ignoring NANs): 0.4999716020776064 percentage of F/O trace that was flagged: 99.93562922433216
Working with slit/slice: 14 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 514 1982] [1628 1648]] Max value in slity array (ignoring NANs): 0.4997611753551952 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 15 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 386 1852] [1372 1394]] Max value in slity array (ignoring NANs): 0.49974517536774626 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 16 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 404 1873] [1856 1880]] Max value in slity array (ignoring NANs): 0.4997112322262246 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 17 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 385 1852] [1512 1534]] Max value in slity array (ignoring NANs): 0.4998204880532977 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 18 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 312 1773] [ 201 221]] Max value in slity array (ignoring NANs): 0.4999855029608435 percentage of F/O trace that was flagged: 99.92138364779875
Working with slit/slice: 19 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 282 1744] [ 619 640]] Max value in slity array (ignoring NANs): 0.49987583080741427 percentage of F/O trace that was flagged: 100.0
*** Final result for msa_flagging test will be reported as PASSED ***
('* MSA flagging validation test took ', '15.403436183929443 seconds to finish.')
Did msa_flagging validation test pass? True
Testing files for detector: nrs2
Working with uncal_file: /data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/mos_g140m_line1_NRS2_uncal.fits
2021-08-27 22:53:11,667 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /grp/crds/cache/references/jwst/jwst_nirspec_pars-detector1pipeline_0004.asdf
2021-08-27 22:53:11,681 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2021-08-27 22:53:11,682 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2021-08-27 22:53:11,683 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2021-08-27 22:53:11,684 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2021-08-27 22:53:11,685 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2021-08-27 22:53:11,686 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2021-08-27 22:53:11,687 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2021-08-27 22:53:11,688 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2021-08-27 22:53:11,689 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2021-08-27 22:53:11,690 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2021-08-27 22:53:11,691 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2021-08-27 22:53:11,692 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2021-08-27 22:53:11,693 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2021-08-27 22:53:11,694 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2021-08-27 22:53:11,695 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2021-08-27 22:53:11,696 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2021-08-27 22:53:11,697 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/stpipe/step.py:359: ResourceWarning: unclosed file <_io.FileIO name='/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/mos_g140m_line1_NRS2_uncal.fits' mode='rb' closefd=True>
gc.collect()
2021-08-27 22:53:11,902 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/mos_g140m_line1_NRS2_uncal.fits',).
2021-08-27 22:53:11,911 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_calibrated_ramp': False, 'steps': {'group_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dq_init': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'saturation': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'ipc': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'superbias': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'refpix': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}, 'rscd': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'type': 'baseline'}, 'firstframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'lastframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'linearity': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dark_current': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'dark_output': None}, 'reset': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'persistence': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'input_trapsfilled': '', 'flag_pers_cutoff': 40.0, 'save_persistence': False, 'save_trapsfilled': True}, 'jump': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}, 'ramp_fit': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}, 'gain_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}}}
2021-08-27 22:53:12,021 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'mos_g140m_line1_NRS2_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'persat', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias', 'trapdensity', 'trappars']
2021-08-27 22:53:12,028 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_dark_0143.fits'.
2021-08-27 22:53:12,030 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_gain_0018.fits'.
2021-08-27 22:53:12,031 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_linearity_0020.fits'.
2021-08-27 22:53:12,032 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_mask_0025.fits'.
2021-08-27 22:53:12,034 - stpipe.Detector1Pipeline - INFO - Prefetch for PERSAT reference file is 'N/A'.
2021-08-27 22:53:12,034 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0016.fits'.
2021-08-27 22:53:12,035 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2021-08-27 22:53:12,035 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2021-08-27 22:53:12,036 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2021-08-27 22:53:12,036 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_saturation_0022.fits'.
2021-08-27 22:53:12,037 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_superbias_0104.fits'.
2021-08-27 22:53:12,038 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPDENSITY reference file is 'N/A'.
2021-08-27 22:53:12,039 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPPARS reference file is 'N/A'.
2021-08-27 22:53:12,039 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2021-08-27 22:53:12,333 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS2_uncal.fits>,).
2021-08-27 22:53:12,334 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:53:12,424 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2021-08-27 22:53:12,425 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2021-08-27 22:53:12,426 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2021-08-27 22:53:12,541 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS2_uncal.fits>,).
2021-08-27 22:53:12,542 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:53:12,560 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /grp/crds/cache/references/jwst/jwst_nirspec_mask_0025.fits
2021-08-27 22:53:12,818 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2021-08-27 22:53:12,937 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS2_uncal.fits>,).
2021-08-27 22:53:12,938 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:53:12,956 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /grp/crds/cache/references/jwst/jwst_nirspec_saturation_0022.fits
2021-08-27 22:53:14,253 - stpipe.Detector1Pipeline.saturation - INFO - Detected 83201 saturated pixels
2021-08-27 22:53:14,284 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2021-08-27 22:53:14,293 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2021-08-27 22:53:14,413 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS2_uncal.fits>,).
2021-08-27 22:53:14,414 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:53:14,414 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2021-08-27 22:53:14,415 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc done
2021-08-27 22:53:14,528 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS2_uncal.fits>,).
2021-08-27 22:53:14,529 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:53:14,547 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /grp/crds/cache/references/jwst/jwst_nirspec_superbias_0104.fits
2021-08-27 22:53:14,798 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2021-08-27 22:53:14,913 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS2_uncal.fits>,).
2021-08-27 22:53:14,915 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}
2021-08-27 22:53:14,923 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2021-08-27 22:53:14,923 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2021-08-27 22:53:14,923 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2021-08-27 22:53:14,924 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.000000
2021-08-27 22:53:14,924 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = True
2021-08-27 22:53:18,214 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2021-08-27 22:53:18,329 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS2_uncal.fits>,).
2021-08-27 22:53:18,330 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:53:18,350 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /grp/crds/cache/references/jwst/jwst_nirspec_linearity_0020.fits
2021-08-27 22:53:22,914 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2021-08-27 22:53:23,035 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS2_uncal.fits>,).
2021-08-27 22:53:23,036 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'dark_output': None}
2021-08-27 22:53:23,056 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /grp/crds/cache/references/jwst/jwst_nirspec_dark_0143.fits
2021-08-27 22:53:23,292 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=6, nframes=1, groupgap=0
2021-08-27 22:53:23,293 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=10, nframes=1, groupgap=0
2021-08-27 22:53:23,451 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2021-08-27 22:53:23,569 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS2_uncal.fits>,).
2021-08-27 22:53:23,571 - stpipe.Detector1Pipeline.jump - INFO - Step jump parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}
2021-08-27 22:53:23,579 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2021-08-27 22:53:23,590 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0018.fits
2021-08-27 22:53:23,642 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0016.fits
2021-08-27 22:53:23,683 - stpipe.Detector1Pipeline.jump - INFO - Using 1 core for jump detection
2021-08-27 22:53:23,874 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2021-08-27 22:53:24,046 - stpipe.Detector1Pipeline.jump - INFO - Working on integration 1:
2021-08-27 22:53:25,635 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 39124 pixels with at least one CR and at least four groups
2021-08-27 22:53:25,636 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 2731 pixels with at least one CR and three groups
2021-08-27 22:53:25,636 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 48084 pixels with at least one CR and two groups
2021-08-27 22:53:28,749 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 4.87382 sec
2021-08-27 22:53:28,751 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 5.171950
2021-08-27 22:53:28,754 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2021-08-27 22:53:28,868 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 6, 2048, 2048) from mos_g140m_line1_NRS2_uncal.fits>,).
2021-08-27 22:53:28,870 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}
2021-08-27 22:53:28,898 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0016.fits
2021-08-27 22:53:28,940 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0018.fits
2021-08-27 22:53:28,978 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = ols
2021-08-27 22:53:28,978 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/jwst/ramp_fitting/ols_fit.py:1169: RuntimeWarning: invalid value encountered in multiply
var_p4[num_int, :, :, :] *= (segs_4[num_int, :, :, :] > 0)
2021-08-27 22:53:51,112 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of groups per integration: 6
2021-08-27 22:53:51,112 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of integrations: 1
2021-08-27 22:53:51,257 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2021-08-27 22:53:51,376 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from mos_g140m_line1_NRS2_uncal.fits>,).
2021-08-27 22:53:51,377 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scale', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:53:51,434 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2021-08-27 22:53:51,435 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2021-08-27 22:53:51,437 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 22:53:51,556 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from mos_g140m_line1_NRS2_uncal.fits>,).
2021-08-27 22:53:51,557 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scaleints', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:53:51,616 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2021-08-27 22:53:51,617 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2021-08-27 22:53:51,619 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 22:53:51,620 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2021-08-27 22:53:51,622 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2021-08-27 22:53:51,631 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 22:53:51,750 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<ImageModel(2048, 2048) from mos_g140m_line1_NRS2_uncal.fits>,).
2021-08-27 22:53:51,751 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.25, 'sip_degree': None, 'sip_max_inv_pix_error': 0.25, 'sip_inv_degree': None, 'sip_npoints': 32, 'slit_y_low': -0.55, 'slit_y_high': 0.55}
V8460001000101_msa.fits
2021-08-27 22:53:51,898 - stpipe.AssignWcsStep - INFO - Retrieving open MSA slitlets for msa_metadata_id = 1 and dither_index = 1
2021-08-27 22:53:52,113 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.1260581910610199 deg
2021-08-27 22:53:52,113 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3316612243652344 deg
2021-08-27 22:53:52,114 - stpipe.AssignWcsStep - INFO - theta_y correction: 0.0002496099796717191 deg
2021-08-27 22:53:52,115 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:53:52,895 - stpipe.AssignWcsStep - INFO - Removing slit 65 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:53:52,908 - stpipe.AssignWcsStep - INFO - Removing slit 66 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:53:52,920 - stpipe.AssignWcsStep - INFO - Removing slit 68 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:53:52,954 - stpipe.AssignWcsStep - INFO - Removing slit 58 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:53:52,966 - stpipe.AssignWcsStep - INFO - Removing slit 59 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:53:52,977 - stpipe.AssignWcsStep - INFO - Removing slit 60 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:53:52,989 - stpipe.AssignWcsStep - INFO - Removing slit 61 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:53:53,001 - stpipe.AssignWcsStep - INFO - Removing slit 62 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:53:53,012 - stpipe.AssignWcsStep - INFO - Removing slit 63 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:53:53,024 - stpipe.AssignWcsStep - INFO - Removing slit 64 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:53:53,036 - stpipe.AssignWcsStep - INFO - Removing slit 67 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:53:53,037 - stpipe.AssignWcsStep - INFO - Slits projected on detector NRS2: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57]
2021-08-27 22:53:53,037 - stpipe.AssignWcsStep - INFO - Computing WCS for 57 open slitlets
2021-08-27 22:53:53,062 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.1260581910610199 deg
2021-08-27 22:53:53,063 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3316612243652344 deg
2021-08-27 22:53:53,063 - stpipe.AssignWcsStep - INFO - theta_y correction: 0.0002496099796717191 deg
2021-08-27 22:53:53,064 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:53:53,073 - stpipe.AssignWcsStep - INFO - SPORDER= -1, wrange=[9.7e-07, 1.89e-06]
2021-08-27 22:53:53,184 - stpipe.AssignWcsStep - INFO - There are 27 open slits in quadrant 1
2021-08-27 22:53:53,524 - stpipe.AssignWcsStep - INFO - There are 27 open slits in quadrant 2
2021-08-27 22:53:53,725 - stpipe.AssignWcsStep - INFO - There are 1 open slits in quadrant 3
2021-08-27 22:53:53,734 - stpipe.AssignWcsStep - INFO - There are 2 open slits in quadrant 4
2021-08-27 22:53:53,749 - stpipe.AssignWcsStep - INFO - There are 0 open slits in quadrant 5
2021-08-27 22:53:53,888 - stpipe.AssignWcsStep - INFO - Created a NIRSPEC nrs_msaspec pipeline with references {'distortion': None, 'filteroffset': None, 'specwcs': None, 'regions': None, 'wavelengthrange': '/grp/crds/cache/references/jwst/jwst_nirspec_wavelengthrange_0004.asdf', 'camera': '/grp/crds/cache/references/jwst/jwst_nirspec_camera_0004.asdf', 'collimator': '/grp/crds/cache/references/jwst/jwst_nirspec_collimator_0004.asdf', 'disperser': '/grp/crds/cache/references/jwst/jwst_nirspec_disperser_0032.asdf', 'fore': '/grp/crds/cache/references/jwst/jwst_nirspec_fore_0023.asdf', 'fpa': '/grp/crds/cache/references/jwst/jwst_nirspec_fpa_0005.asdf', 'msa': '/grp/crds/cache/references/jwst/jwst_nirspec_msa_0005.asdf', 'ote': '/grp/crds/cache/references/jwst/jwst_nirspec_ote_0005.asdf', 'ifupost': None, 'ifufore': None, 'ifuslicer': None, 'msametafile': 'V8460001000101_msa.fits'}
2021-08-27 22:53:54,791 - stpipe.AssignWcsStep - INFO - COMPLETED assign_wcs
2021-08-27 22:53:54,797 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep done
2021-08-27 22:53:54,806 - stpipe.MSAFlagOpenStep - INFO - MSAFlagOpenStep instance created.
2021-08-27 22:53:54,983 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep running with args (<ImageModel(2048, 2048) from mos_g140m_line1_NRS2_uncal.fits>,).
2021-08-27 22:53:54,984 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}
2021-08-27 22:53:55,000 - stpipe.MSAFlagOpenStep - INFO - Using reference file /grp/crds/cache/references/jwst/jwst_nirspec_msaoper_0001.json
2021-08-27 22:53:55,001 - stpipe.JwstStep - INFO - JwstStep instance created.
2021-08-27 22:53:55,113 - stpipe.MSAFlagOpenStep - INFO - gwa_ytilt is 0.1260581910610199 deg
2021-08-27 22:53:55,113 - stpipe.MSAFlagOpenStep - INFO - gwa_xtilt is 0.3316612243652344 deg
2021-08-27 22:53:55,114 - stpipe.MSAFlagOpenStep - INFO - theta_y correction: 0.0002496099796717191 deg
2021-08-27 22:53:55,115 - stpipe.MSAFlagOpenStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:53:55,123 - stpipe.MSAFlagOpenStep - INFO - SPORDER= -1, wrange=[9.7e-07, 1.89e-06]
2021-08-27 22:53:55,232 - stpipe.MSAFlagOpenStep - INFO - There are 5 open slits in quadrant 1
2021-08-27 22:53:55,270 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 2
2021-08-27 22:53:55,293 - stpipe.MSAFlagOpenStep - INFO - There are 9 open slits in quadrant 3
2021-08-27 22:53:55,361 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 4
2021-08-27 22:53:55,384 - stpipe.MSAFlagOpenStep - INFO - There are 0 open slits in quadrant 5
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/gwcs/geometry.py:203: RuntimeWarning: invalid value encountered in remainder
lon = np.mod(lon, 360.0 * u.deg if nquant else 360.0)
2021-08-27 22:54:07,510 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep done
2021-08-27 22:54:08,172 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 22:54:08,182 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 22:54:08,575 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<ImageModel(2048, 2048) from mos_g140m_line1_NRS2_uncal.fits>,).
2021-08-27 22:54:08,577 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.25, 'sip_degree': None, 'sip_max_inv_pix_error': 0.25, 'sip_inv_degree': None, 'sip_npoints': 32, 'slit_y_low': -0.45, 'slit_y_high': 0.45}
2021-08-27 22:54:08,707 - stpipe.AssignWcsStep - INFO - Retrieving open MSA slitlets for msa_metadata_id = 1 and dither_index = 1
2021-08-27 22:54:08,755 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.1260581910610199 deg
2021-08-27 22:54:08,755 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3316612243652344 deg
2021-08-27 22:54:08,756 - stpipe.AssignWcsStep - INFO - theta_y correction: 0.0002496099796717191 deg
2021-08-27 22:54:08,757 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:54:09,033 - stpipe.AssignWcsStep - INFO - Removing slit 13 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:54:09,045 - stpipe.AssignWcsStep - INFO - Removing slit 14 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:54:09,057 - stpipe.AssignWcsStep - INFO - Removing slit 15 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:54:09,068 - stpipe.AssignWcsStep - INFO - Removing slit 16 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:54:09,080 - stpipe.AssignWcsStep - INFO - Removing slit 17 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:54:09,092 - stpipe.AssignWcsStep - INFO - Removing slit 18 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:54:09,104 - stpipe.AssignWcsStep - INFO - Removing slit 19 from the list of open slits because the WCS bounding_box is completely outside the detector.
2021-08-27 22:54:09,104 - stpipe.AssignWcsStep - INFO - Slits projected on detector NRS2: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
2021-08-27 22:54:09,105 - stpipe.AssignWcsStep - INFO - Computing WCS for 12 open slitlets
2021-08-27 22:54:09,130 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.1260581910610199 deg
2021-08-27 22:54:09,131 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3316612243652344 deg
2021-08-27 22:54:09,131 - stpipe.AssignWcsStep - INFO - theta_y correction: 0.0002496099796717191 deg
2021-08-27 22:54:09,132 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:54:09,141 - stpipe.AssignWcsStep - INFO - SPORDER= -1, wrange=[9.7e-07, 1.89e-06]
2021-08-27 22:54:09,251 - stpipe.AssignWcsStep - INFO - There are 5 open slits in quadrant 1
2021-08-27 22:54:09,289 - stpipe.AssignWcsStep - INFO - There are 3 open slits in quadrant 2
2021-08-27 22:54:09,313 - stpipe.AssignWcsStep - INFO - There are 4 open slits in quadrant 3
2021-08-27 22:54:09,345 - stpipe.AssignWcsStep - INFO - There are 0 open slits in quadrant 4
2021-08-27 22:54:09,345 - stpipe.AssignWcsStep - INFO - There are 0 open slits in quadrant 5
2021-08-27 22:54:09,463 - stpipe.AssignWcsStep - INFO - Created a NIRSPEC nrs_msaspec pipeline with references {'distortion': None, 'filteroffset': None, 'specwcs': None, 'regions': None, 'wavelengthrange': '/grp/crds/cache/references/jwst/jwst_nirspec_wavelengthrange_0004.asdf', 'camera': '/grp/crds/cache/references/jwst/jwst_nirspec_camera_0004.asdf', 'collimator': '/grp/crds/cache/references/jwst/jwst_nirspec_collimator_0004.asdf', 'disperser': '/grp/crds/cache/references/jwst/jwst_nirspec_disperser_0032.asdf', 'fore': '/grp/crds/cache/references/jwst/jwst_nirspec_fore_0023.asdf', 'fpa': '/grp/crds/cache/references/jwst/jwst_nirspec_fpa_0005.asdf', 'msa': '/grp/crds/cache/references/jwst/jwst_nirspec_msa_0005.asdf', 'ote': '/grp/crds/cache/references/jwst/jwst_nirspec_ote_0005.asdf', 'ifupost': None, 'ifufore': None, 'ifuslicer': None, 'msametafile': 'fopens_metafile_msa.fits'}
2021-08-27 22:54:09,717 - stpipe.AssignWcsStep - INFO - COMPLETED assign_wcs
2021-08-27 22:54:09,723 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep done
Working with slit/slice: 1 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 131 1613] [1906 1930]] Max value in slity array (ignoring NANs): 0.499879629054766 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 2 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 0 1330] [1823 1842]] Max value in slity array (ignoring NANs): 0.49995226747347604 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 3 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 0 957] [1631 1645]] Max value in slity array (ignoring NANs): 0.4997648773778204 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 4 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 0 906] [1320 1334]] Max value in slity array (ignoring NANs): 0.49972723641802796 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 5 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 0 814] [1591 1604]] Max value in slity array (ignoring NANs): 0.4997225330923289 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 6 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 50 1525] [ 680 699]] Max value in slity array (ignoring NANs): 0.49989116447537363 percentage of F/O trace that was flagged: 99.93728441517717
Working with slit/slice: 7 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 0 804] [239 251]] Max value in slity array (ignoring NANs): 0.49975066484744846 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 8 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 0 783] [163 175]] Max value in slity array (ignoring NANs): 0.49955175410051833 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 9 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 0 388] [1260 1273]] Max value in slity array (ignoring NANs): 0.4996146477571852 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 10 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 0 387] [1230 1243]] Max value in slity array (ignoring NANs): 0.4993629738181582 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 11 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 0 157] [1825 1841]] Max value in slity array (ignoring NANs): 0.35020493940891617 percentage of F/O trace that was flagged: 100.0
Working with slit/slice: 12 Slit min and max in y direction: -0.5 0.5 bounding box rounded to next integer: [[ 0 94] [1249 1265]] Max value in slity array (ignoring NANs): 0.4435073830894076 percentage of F/O trace that was flagged: 100.0
*** Final result for msa_flagging test will be reported as PASSED ***
('* MSA flagging validation test took ', '10.860138177871704 seconds to finish.')
Did msa_flagging validation test pass? True
Testing files for detector: nrs1
Working with uncal_file: /data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/ifu_prism_nrs1_uncal.fits
2021-08-27 22:54:19,278 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /grp/crds/cache/references/jwst/jwst_nirspec_pars-detector1pipeline_0004.asdf
2021-08-27 22:54:19,292 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2021-08-27 22:54:19,293 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2021-08-27 22:54:19,294 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2021-08-27 22:54:19,295 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2021-08-27 22:54:19,296 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2021-08-27 22:54:19,297 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2021-08-27 22:54:19,298 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2021-08-27 22:54:19,299 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2021-08-27 22:54:19,300 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2021-08-27 22:54:19,301 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2021-08-27 22:54:19,302 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2021-08-27 22:54:19,303 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2021-08-27 22:54:19,304 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2021-08-27 22:54:19,305 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2021-08-27 22:54:19,306 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2021-08-27 22:54:19,307 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2021-08-27 22:54:19,308 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/stpipe/step.py:359: ResourceWarning: unclosed file <_io.FileIO name='/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/ifu_prism_nrs1_uncal.fits' mode='rb' closefd=True>
gc.collect()
2021-08-27 22:54:19,527 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/ifu_prism_nrs1_uncal.fits',).
2021-08-27 22:54:19,535 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_calibrated_ramp': False, 'steps': {'group_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dq_init': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'saturation': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'ipc': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'superbias': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'refpix': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}, 'rscd': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'type': 'baseline'}, 'firstframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'lastframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'linearity': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dark_current': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'dark_output': None}, 'reset': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'persistence': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'input_trapsfilled': '', 'flag_pers_cutoff': 40.0, 'save_persistence': False, 'save_trapsfilled': True}, 'jump': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}, 'ramp_fit': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}, 'gain_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}}}
2021-08-27 22:54:19,672 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'ifu_prism_nrs1_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'persat', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias', 'trapdensity', 'trappars']
2021-08-27 22:54:19,680 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_dark_0140.fits'.
2021-08-27 22:54:19,681 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_gain_0019.fits'.
2021-08-27 22:54:19,682 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_linearity_0018.fits'.
2021-08-27 22:54:19,684 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_mask_0024.fits'.
2021-08-27 22:54:19,685 - stpipe.Detector1Pipeline - INFO - Prefetch for PERSAT reference file is 'N/A'.
2021-08-27 22:54:19,685 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0019.fits'.
2021-08-27 22:54:19,687 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2021-08-27 22:54:19,687 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2021-08-27 22:54:19,687 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2021-08-27 22:54:19,687 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_saturation_0020.fits'.
2021-08-27 22:54:19,689 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_superbias_0087.fits'.
2021-08-27 22:54:19,690 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPDENSITY reference file is 'N/A'.
2021-08-27 22:54:19,690 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPPARS reference file is 'N/A'.
2021-08-27 22:54:19,691 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2021-08-27 22:54:20,045 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs1_uncal.fits>,).
2021-08-27 22:54:20,047 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:54:20,163 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2021-08-27 22:54:20,164 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2021-08-27 22:54:20,165 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2021-08-27 22:54:20,302 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs1_uncal.fits>,).
2021-08-27 22:54:20,304 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:54:20,321 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /grp/crds/cache/references/jwst/jwst_nirspec_mask_0024.fits
2021-08-27 22:54:20,558 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2021-08-27 22:54:20,699 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs1_uncal.fits>,).
2021-08-27 22:54:20,701 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:54:20,719 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /grp/crds/cache/references/jwst/jwst_nirspec_saturation_0020.fits
2021-08-27 22:54:21,236 - stpipe.Detector1Pipeline.saturation - INFO - Detected 4320 saturated pixels
2021-08-27 22:54:21,280 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2021-08-27 22:54:21,288 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2021-08-27 22:54:21,429 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs1_uncal.fits>,).
2021-08-27 22:54:21,431 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:54:21,431 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2021-08-27 22:54:21,432 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc done
2021-08-27 22:54:21,569 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs1_uncal.fits>,).
2021-08-27 22:54:21,570 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:54:21,589 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /grp/crds/cache/references/jwst/jwst_nirspec_superbias_0087.fits
2021-08-27 22:54:21,871 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2021-08-27 22:54:22,009 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs1_uncal.fits>,).
2021-08-27 22:54:22,011 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}
2021-08-27 22:54:22,018 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2021-08-27 22:54:22,019 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2021-08-27 22:54:22,019 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2021-08-27 22:54:22,020 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.000000
2021-08-27 22:54:22,020 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = True
2021-08-27 22:54:26,540 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2021-08-27 22:54:26,684 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs1_uncal.fits>,).
2021-08-27 22:54:26,685 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:54:26,704 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /grp/crds/cache/references/jwst/jwst_nirspec_linearity_0018.fits
2021-08-27 22:54:27,586 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2021-08-27 22:54:27,727 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs1_uncal.fits>,).
2021-08-27 22:54:27,728 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'dark_output': None}
2021-08-27 22:54:27,746 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /grp/crds/cache/references/jwst/jwst_nirspec_dark_0140.fits
2021-08-27 22:54:27,946 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=10, nframes=1, groupgap=0
2021-08-27 22:54:27,947 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=10, nframes=1, groupgap=0
2021-08-27 22:54:28,229 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2021-08-27 22:54:28,373 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs1_uncal.fits>,).
2021-08-27 22:54:28,374 - stpipe.Detector1Pipeline.jump - INFO - Step jump parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}
2021-08-27 22:54:28,382 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2021-08-27 22:54:28,393 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0019.fits
2021-08-27 22:54:28,441 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0019.fits
2021-08-27 22:54:28,482 - stpipe.Detector1Pipeline.jump - INFO - Using 1 core for jump detection
2021-08-27 22:54:28,704 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2021-08-27 22:54:28,969 - stpipe.Detector1Pipeline.jump - INFO - Working on integration 1:
2021-08-27 22:54:31,426 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 75673 pixels with at least one CR and at least four groups
2021-08-27 22:54:31,427 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 4 pixels with at least one CR and three groups
2021-08-27 22:54:31,428 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 4093 pixels with at least one CR and two groups
2021-08-27 22:54:33,422 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 4.7169 sec
2021-08-27 22:54:33,424 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 5.042202
2021-08-27 22:54:33,428 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2021-08-27 22:54:33,584 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs1_uncal.fits>,).
2021-08-27 22:54:33,585 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}
2021-08-27 22:54:33,613 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0019.fits
2021-08-27 22:54:33,660 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0019.fits
2021-08-27 22:54:33,700 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = ols
2021-08-27 22:54:33,700 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/jwst/ramp_fitting/ols_fit.py:1169: RuntimeWarning: invalid value encountered in multiply
var_p4[num_int, :, :, :] *= (segs_4[num_int, :, :, :] > 0)
2021-08-27 22:55:17,668 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of groups per integration: 10
2021-08-27 22:55:17,669 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of integrations: 1
2021-08-27 22:55:17,900 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2021-08-27 22:55:18,054 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<IFUImageModel(2048, 2048) from ifu_prism_nrs1_uncal.fits>,).
2021-08-27 22:55:18,056 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scale', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:55:18,115 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2021-08-27 22:55:18,116 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2021-08-27 22:55:18,118 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 22:55:18,266 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from ifu_prism_nrs1_uncal.fits>,).
2021-08-27 22:55:18,268 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scaleints', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:55:18,323 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2021-08-27 22:55:18,324 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2021-08-27 22:55:18,326 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 22:55:18,327 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2021-08-27 22:55:18,328 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2021-08-27 22:55:18,338 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 22:55:18,485 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<IFUImageModel(2048, 2048) from ifu_prism_nrs1_uncal.fits>,).
2021-08-27 22:55:18,487 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.25, 'sip_degree': None, 'sip_max_inv_pix_error': 0.25, 'sip_inv_degree': None, 'sip_npoints': 32, 'slit_y_low': -0.55, 'slit_y_high': 0.55}
2021-08-27 22:55:18,660 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.03312480077147484 deg
2021-08-27 22:55:18,661 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3411945700645447 deg
2021-08-27 22:55:18,662 - stpipe.AssignWcsStep - INFO - theta_y correction: -0.005294283663966503 deg
2021-08-27 22:55:18,662 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:55:21,810 - stpipe.AssignWcsStep - INFO - Created a NIRSPEC nrs_ifu pipeline with references {'distortion': None, 'filteroffset': None, 'specwcs': None, 'regions': None, 'wavelengthrange': '/grp/crds/cache/references/jwst/jwst_nirspec_wavelengthrange_0004.asdf', 'camera': '/grp/crds/cache/references/jwst/jwst_nirspec_camera_0004.asdf', 'collimator': '/grp/crds/cache/references/jwst/jwst_nirspec_collimator_0004.asdf', 'disperser': '/grp/crds/cache/references/jwst/jwst_nirspec_disperser_0034.asdf', 'fore': '/grp/crds/cache/references/jwst/jwst_nirspec_fore_0028.asdf', 'fpa': '/grp/crds/cache/references/jwst/jwst_nirspec_fpa_0005.asdf', 'msa': '/grp/crds/cache/references/jwst/jwst_nirspec_msa_0005.asdf', 'ote': '/grp/crds/cache/references/jwst/jwst_nirspec_ote_0005.asdf', 'ifupost': '/grp/crds/cache/references/jwst/jwst_nirspec_ifupost_0004.asdf', 'ifufore': '/grp/crds/cache/references/jwst/jwst_nirspec_ifufore_0003.asdf', 'ifuslicer': '/grp/crds/cache/references/jwst/jwst_nirspec_ifuslicer_0003.asdf'}
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/gwcs/geometry.py:203: RuntimeWarning: invalid value encountered in remainder
lon = np.mod(lon, 360.0 * u.deg if nquant else 360.0)
2021-08-27 22:55:26,273 - stpipe.AssignWcsStep - INFO - Update S_REGION to POLYGON ICRS 156.176998032 -45.687618307 156.178780407 -45.687618307 156.178780407 -45.686330486 156.176998032 -45.686330486
2021-08-27 22:55:26,275 - stpipe.AssignWcsStep - INFO - COMPLETED assign_wcs
2021-08-27 22:55:26,281 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep done
2021-08-27 22:55:26,291 - stpipe.MSAFlagOpenStep - INFO - MSAFlagOpenStep instance created.
2021-08-27 22:55:26,587 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep running with args (<IFUImageModel(2048, 2048) from ifu_prism_nrs1_uncal.fits>,).
2021-08-27 22:55:26,589 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}
2021-08-27 22:55:26,607 - stpipe.MSAFlagOpenStep - INFO - Using reference file /grp/crds/cache/references/jwst/jwst_nirspec_msaoper_0001.json
2021-08-27 22:55:26,608 - stpipe.JwstStep - INFO - JwstStep instance created.
2021-08-27 22:55:26,721 - stpipe.MSAFlagOpenStep - INFO - gwa_ytilt is 0.03312480077147484 deg
2021-08-27 22:55:26,721 - stpipe.MSAFlagOpenStep - INFO - gwa_xtilt is 0.3411945700645447 deg
2021-08-27 22:55:26,722 - stpipe.MSAFlagOpenStep - INFO - theta_y correction: -0.005294283663966503 deg
2021-08-27 22:55:26,723 - stpipe.MSAFlagOpenStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:55:26,731 - stpipe.MSAFlagOpenStep - INFO - SPORDER= 0, wrange=[6e-07, 5.3e-06]
2021-08-27 22:55:26,844 - stpipe.MSAFlagOpenStep - INFO - There are 5 open slits in quadrant 1
2021-08-27 22:55:26,883 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 2
2021-08-27 22:55:26,907 - stpipe.MSAFlagOpenStep - INFO - There are 9 open slits in quadrant 3
2021-08-27 22:55:26,979 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 4
2021-08-27 22:55:27,003 - stpipe.MSAFlagOpenStep - INFO - There are 0 open slits in quadrant 5
2021-08-27 22:55:40,272 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep done
2021-08-27 22:55:40,891 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 22:55:40,902 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 22:55:41,444 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<IFUImageModel(2048, 2048) from ifu_prism_nrs1_uncal.fits>,).
2021-08-27 22:55:41,446 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.25, 'sip_degree': None, 'sip_max_inv_pix_error': 0.25, 'sip_inv_degree': None, 'sip_npoints': 32, 'slit_y_low': -0.45, 'slit_y_high': 0.45}
2021-08-27 22:55:41,604 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.03312480077147484 deg
2021-08-27 22:55:41,605 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3411945700645447 deg
2021-08-27 22:55:41,605 - stpipe.AssignWcsStep - INFO - theta_y correction: -0.005294283663966503 deg
2021-08-27 22:55:41,606 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 22:55:44,945 - stpipe.AssignWcsStep - INFO - Created a NIRSPEC nrs_ifu pipeline with references {'distortion': None, 'filteroffset': None, 'specwcs': None, 'regions': None, 'wavelengthrange': '/grp/crds/cache/references/jwst/jwst_nirspec_wavelengthrange_0004.asdf', 'camera': '/grp/crds/cache/references/jwst/jwst_nirspec_camera_0004.asdf', 'collimator': '/grp/crds/cache/references/jwst/jwst_nirspec_collimator_0004.asdf', 'disperser': '/grp/crds/cache/references/jwst/jwst_nirspec_disperser_0034.asdf', 'fore': '/grp/crds/cache/references/jwst/jwst_nirspec_fore_0028.asdf', 'fpa': '/grp/crds/cache/references/jwst/jwst_nirspec_fpa_0005.asdf', 'msa': '/grp/crds/cache/references/jwst/jwst_nirspec_msa_0005.asdf', 'ote': '/grp/crds/cache/references/jwst/jwst_nirspec_ote_0005.asdf', 'ifupost': '/grp/crds/cache/references/jwst/jwst_nirspec_ifupost_0004.asdf', 'ifufore': '/grp/crds/cache/references/jwst/jwst_nirspec_ifufore_0003.asdf', 'ifuslicer': '/grp/crds/cache/references/jwst/jwst_nirspec_ifuslicer_0003.asdf'}
2021-08-27 22:55:49,323 - stpipe.AssignWcsStep - INFO - Update S_REGION to POLYGON ICRS 156.177072264 -45.687546534 156.178706424 -45.687546534 156.178706424 -45.686402088 156.177072264 -45.686402088
2021-08-27 22:55:49,324 - stpipe.AssignWcsStep - INFO - COMPLETED assign_wcs
2021-08-27 22:55:49,331 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep done
Working with slit/slice: 0 bounding box rounded to next integer: [[561 998] [779 818]] Max value in slity array (ignoring NANs): 0.44998705348588053 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 1 bounding box rounded to next integer: [[ 505 946] [1196 1237]] Max value in slity array (ignoring NANs): 0.4499843337191686 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 2 bounding box rounded to next integer: [[ 567 1004] [ 729 769]] Max value in slity array (ignoring NANs): 0.44987649498755533 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 3 bounding box rounded to next integer: [[ 498 939] [1245 1286]] Max value in slity array (ignoring NANs): 0.4499849195730806 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 4 bounding box rounded to next integer: [[ 573 1010] [ 680 720]] Max value in slity array (ignoring NANs): 0.4499417148765048 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 5 bounding box rounded to next integer: [[ 491 932] [1294 1335]] Max value in slity array (ignoring NANs): 0.4499544968703606 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 6 bounding box rounded to next integer: [[ 579 1016] [ 631 670]] Max value in slity array (ignoring NANs): 0.44995341126003097 percentage of F/O trace that was flagged: 9.974511469838571
Working with slit/slice: 7 bounding box rounded to next integer: [[ 484 926] [1344 1384]] Max value in slity array (ignoring NANs): 0.4499996134104647 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 8 bounding box rounded to next integer: [[ 585 1021] [ 581 621]] Max value in slity array (ignoring NANs): 0.44999451028248455 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 9 bounding box rounded to next integer: [[ 477 919] [1393 1433]] Max value in slity array (ignoring NANs): 0.4498616000474087 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 10 bounding box rounded to next integer: [[ 591 1027] [ 532 572]] Max value in slity array (ignoring NANs): 0.44996048651820686 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 11 bounding box rounded to next integer: [[ 469 912] [1442 1483]] Max value in slity array (ignoring NANs): 0.4499575501049847 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 12 bounding box rounded to next integer: [[ 597 1032] [ 483 522]] Max value in slity array (ignoring NANs): 0.44990591840895894 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 13 bounding box rounded to next integer: [[ 462 905] [1491 1532]] Max value in slity array (ignoring NANs): 0.4498447168478521 percentage of F/O trace that was flagged: 4.279661016949152
Working with slit/slice: 14 bounding box rounded to next integer: [[ 603 1038] [ 433 473]] Max value in slity array (ignoring NANs): 0.4499878283300202 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 15 bounding box rounded to next integer: [[ 455 898] [1540 1581]] Max value in slity array (ignoring NANs): 0.44979571721669287 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 16 bounding box rounded to next integer: [[ 608 1043] [ 384 424]] Max value in slity array (ignoring NANs): 0.4499677044855765 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 17 bounding box rounded to next integer: [[ 447 891] [1589 1630]] Max value in slity array (ignoring NANs): 0.4499355040379717 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 18 bounding box rounded to next integer: [[ 614 1048] [ 335 374]] Max value in slity array (ignoring NANs): 0.4499491042319192 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 19 bounding box rounded to next integer: [[ 440 884] [1638 1679]] Max value in slity array (ignoring NANs): 0.4498325737274414 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 20 bounding box rounded to next integer: [[ 619 1054] [ 285 325]] Max value in slity array (ignoring NANs): 0.4499255242745253 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 21 bounding box rounded to next integer: [[ 432 876] [1687 1728]] Max value in slity array (ignoring NANs): 0.44994879398919385 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 22 bounding box rounded to next integer: [[ 625 1059] [ 236 276]] Max value in slity array (ignoring NANs): 0.44990723924088283 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 23 bounding box rounded to next integer: [[ 424 869] [1736 1777]] Max value in slity array (ignoring NANs): 0.44997783032577415 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 24 bounding box rounded to next integer: [[ 630 1064] [ 186 226]] Max value in slity array (ignoring NANs): 0.4499791123620826 percentage of F/O trace that was flagged: 8.003718728870858
Working with slit/slice: 25 bounding box rounded to next integer: [[ 416 862] [1784 1826]] Max value in slity array (ignoring NANs): 0.44994712583049956 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 26 bounding box rounded to next integer: [[ 635 1069] [ 137 177]] Max value in slity array (ignoring NANs): 0.44994230691007925 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 27 bounding box rounded to next integer: [[ 408 854] [1833 1875]] Max value in slity array (ignoring NANs): 0.44996900105926085 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 28 bounding box rounded to next integer: [[ 640 1074] [ 87 127]] Max value in slity array (ignoring NANs): 0.44994336903862603 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 29 bounding box rounded to next integer: [[ 401 847] [1882 1924]] Max value in slity array (ignoring NANs): 0.44988528941395867 percentage of F/O trace that was flagged: 0.0
*** Final result for msa_flagging test will be reported as FAILED ***
('* MSA flagging validation test took ', '1.436973253885905 minutes to finish.')
Did msa_flagging validation test pass? False
Testing files for detector: nrs2
Working with uncal_file: /data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/ifu_prism_nrs2_uncal.fits
2021-08-27 22:57:07,486 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /grp/crds/cache/references/jwst/jwst_nirspec_pars-detector1pipeline_0004.asdf
2021-08-27 22:57:07,500 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2021-08-27 22:57:07,501 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2021-08-27 22:57:07,502 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2021-08-27 22:57:07,503 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2021-08-27 22:57:07,505 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2021-08-27 22:57:07,505 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2021-08-27 22:57:07,506 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2021-08-27 22:57:07,507 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2021-08-27 22:57:07,508 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2021-08-27 22:57:07,509 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2021-08-27 22:57:07,510 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2021-08-27 22:57:07,511 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2021-08-27 22:57:07,512 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2021-08-27 22:57:07,513 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2021-08-27 22:57:07,514 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2021-08-27 22:57:07,515 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2021-08-27 22:57:07,517 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/stpipe/step.py:359: ResourceWarning: unclosed file <_io.FileIO name='/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/ifu_prism_nrs2_uncal.fits' mode='rb' closefd=True>
gc.collect()
2021-08-27 22:57:07,973 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/ifu_prism_nrs2_uncal.fits',).
2021-08-27 22:57:07,983 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_calibrated_ramp': False, 'steps': {'group_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dq_init': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'saturation': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'ipc': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'superbias': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'refpix': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}, 'rscd': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'type': 'baseline'}, 'firstframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'lastframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'linearity': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dark_current': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'dark_output': None}, 'reset': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'persistence': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'input_trapsfilled': '', 'flag_pers_cutoff': 40.0, 'save_persistence': False, 'save_trapsfilled': True}, 'jump': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}, 'ramp_fit': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}, 'gain_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}}}
2021-08-27 22:57:08,107 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'ifu_prism_nrs2_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'persat', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias', 'trapdensity', 'trappars']
2021-08-27 22:57:08,115 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_dark_0143.fits'.
2021-08-27 22:57:08,116 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_gain_0018.fits'.
2021-08-27 22:57:08,117 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_linearity_0021.fits'.
2021-08-27 22:57:08,119 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_mask_0025.fits'.
2021-08-27 22:57:08,120 - stpipe.Detector1Pipeline - INFO - Prefetch for PERSAT reference file is 'N/A'.
2021-08-27 22:57:08,120 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0016.fits'.
2021-08-27 22:57:08,121 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2021-08-27 22:57:08,121 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2021-08-27 22:57:08,122 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2021-08-27 22:57:08,122 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_saturation_0021.fits'.
2021-08-27 22:57:08,123 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_superbias_0104.fits'.
2021-08-27 22:57:08,125 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPDENSITY reference file is 'N/A'.
2021-08-27 22:57:08,125 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPPARS reference file is 'N/A'.
2021-08-27 22:57:08,125 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2021-08-27 22:57:08,629 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs2_uncal.fits>,).
2021-08-27 22:57:08,631 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:57:08,895 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2021-08-27 22:57:08,896 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2021-08-27 22:57:08,897 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2021-08-27 22:57:09,054 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs2_uncal.fits>,).
2021-08-27 22:57:09,055 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:57:09,073 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /grp/crds/cache/references/jwst/jwst_nirspec_mask_0025.fits
2021-08-27 22:57:09,356 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2021-08-27 22:57:09,516 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs2_uncal.fits>,).
2021-08-27 22:57:09,518 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:57:09,536 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /grp/crds/cache/references/jwst/jwst_nirspec_saturation_0021.fits
2021-08-27 22:57:10,054 - stpipe.Detector1Pipeline.saturation - INFO - Detected 2388 saturated pixels
2021-08-27 22:57:10,087 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2021-08-27 22:57:10,095 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2021-08-27 22:57:10,248 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs2_uncal.fits>,).
2021-08-27 22:57:10,249 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:57:10,250 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2021-08-27 22:57:10,251 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc done
2021-08-27 22:57:10,401 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs2_uncal.fits>,).
2021-08-27 22:57:10,402 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:57:10,421 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /grp/crds/cache/references/jwst/jwst_nirspec_superbias_0104.fits
2021-08-27 22:57:10,696 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2021-08-27 22:57:10,850 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs2_uncal.fits>,).
2021-08-27 22:57:10,852 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}
2021-08-27 22:57:10,860 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2021-08-27 22:57:10,860 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2021-08-27 22:57:10,860 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2021-08-27 22:57:10,860 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.000000
2021-08-27 22:57:10,861 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = True
2021-08-27 22:57:14,988 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2021-08-27 22:57:15,168 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs2_uncal.fits>,).
2021-08-27 22:57:15,170 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:57:15,188 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /grp/crds/cache/references/jwst/jwst_nirspec_linearity_0021.fits
2021-08-27 22:57:16,078 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2021-08-27 22:57:16,237 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs2_uncal.fits>,).
2021-08-27 22:57:16,238 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'dark_output': None}
2021-08-27 22:57:16,256 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /grp/crds/cache/references/jwst/jwst_nirspec_dark_0143.fits
2021-08-27 22:57:16,478 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=10, nframes=1, groupgap=0
2021-08-27 22:57:16,478 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=10, nframes=1, groupgap=0
2021-08-27 22:57:16,674 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2021-08-27 22:57:16,842 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs2_uncal.fits>,).
2021-08-27 22:57:16,843 - stpipe.Detector1Pipeline.jump - INFO - Step jump parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}
2021-08-27 22:57:16,851 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2021-08-27 22:57:16,861 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0018.fits
2021-08-27 22:57:16,908 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0016.fits
2021-08-27 22:57:16,945 - stpipe.Detector1Pipeline.jump - INFO - Using 1 core for jump detection
2021-08-27 22:57:17,159 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2021-08-27 22:57:17,423 - stpipe.Detector1Pipeline.jump - INFO - Working on integration 1:
2021-08-27 22:57:19,967 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 28387 pixels with at least one CR and at least four groups
2021-08-27 22:57:19,969 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 0 pixels with at least one CR and three groups
2021-08-27 22:57:19,969 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 2294 pixels with at least one CR and two groups
2021-08-27 22:57:20,912 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 3.75262 sec
2021-08-27 22:57:20,915 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 4.063860
2021-08-27 22:57:20,917 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2021-08-27 22:57:21,082 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 10, 2048, 2048) from ifu_prism_nrs2_uncal.fits>,).
2021-08-27 22:57:21,084 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}
2021-08-27 22:57:21,112 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0016.fits
2021-08-27 22:57:21,150 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0018.fits
2021-08-27 22:57:21,188 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = ols
2021-08-27 22:57:21,188 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/jwst/ramp_fitting/ols_fit.py:1169: RuntimeWarning: invalid value encountered in multiply
var_p4[num_int, :, :, :] *= (segs_4[num_int, :, :, :] > 0)
2021-08-27 22:57:59,036 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of groups per integration: 10
2021-08-27 22:57:59,037 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of integrations: 1
2021-08-27 22:57:59,259 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2021-08-27 22:57:59,422 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<IFUImageModel(2048, 2048) from ifu_prism_nrs2_uncal.fits>,).
2021-08-27 22:57:59,423 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scale', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:57:59,480 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2021-08-27 22:57:59,480 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2021-08-27 22:57:59,482 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 22:57:59,638 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from ifu_prism_nrs2_uncal.fits>,).
2021-08-27 22:57:59,639 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scaleints', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:57:59,694 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2021-08-27 22:57:59,695 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2021-08-27 22:57:59,697 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 22:57:59,698 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2021-08-27 22:57:59,699 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2021-08-27 22:57:59,708 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 22:57:59,868 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<IFUImageModel(2048, 2048) from ifu_prism_nrs2_uncal.fits>,).
2021-08-27 22:57:59,870 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.25, 'sip_degree': None, 'sip_max_inv_pix_error': 0.25, 'sip_inv_degree': None, 'sip_npoints': 32, 'slit_y_low': -0.55, 'slit_y_high': 0.55}
2021-08-27 22:58:00,012 - stpipe.AssignWcsStep - CRITICAL - No IFU slices fall on detector NRS2
An error occured with AssignWcs. Likely: No open slits fall on detector nrs2 Skipping test for this file. Did msa_flagging validation test pass? skipped Testing files for detector: nrs1 Working with uncal_file: /data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/ifu_g395h_f290lp_nrs1_uncal.fits
2021-08-27 22:58:02,283 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /grp/crds/cache/references/jwst/jwst_nirspec_pars-detector1pipeline_0004.asdf
2021-08-27 22:58:02,296 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2021-08-27 22:58:02,297 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2021-08-27 22:58:02,298 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2021-08-27 22:58:02,299 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2021-08-27 22:58:02,300 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2021-08-27 22:58:02,301 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2021-08-27 22:58:02,302 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2021-08-27 22:58:02,303 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2021-08-27 22:58:02,304 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2021-08-27 22:58:02,305 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2021-08-27 22:58:02,306 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2021-08-27 22:58:02,307 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2021-08-27 22:58:02,308 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2021-08-27 22:58:02,309 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2021-08-27 22:58:02,310 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2021-08-27 22:58:02,311 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2021-08-27 22:58:02,313 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/stpipe/step.py:359: ResourceWarning: unclosed file <_io.FileIO name='/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/ifu_g395h_f290lp_nrs1_uncal.fits' mode='rb' closefd=True>
gc.collect()
2021-08-27 22:58:02,473 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/ifu_g395h_f290lp_nrs1_uncal.fits',).
2021-08-27 22:58:02,482 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_calibrated_ramp': False, 'steps': {'group_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dq_init': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'saturation': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'ipc': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'superbias': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'refpix': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}, 'rscd': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'type': 'baseline'}, 'firstframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'lastframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'linearity': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dark_current': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'dark_output': None}, 'reset': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'persistence': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'input_trapsfilled': '', 'flag_pers_cutoff': 40.0, 'save_persistence': False, 'save_trapsfilled': True}, 'jump': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}, 'ramp_fit': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}, 'gain_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}}}
2021-08-27 22:58:02,728 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'ifu_g395h_f290lp_nrs1_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'persat', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias', 'trapdensity', 'trappars']
2021-08-27 22:58:02,736 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_dark_0088.fits'.
2021-08-27 22:58:02,737 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_gain_0023.fits'.
2021-08-27 22:58:02,739 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_linearity_0022.fits'.
2021-08-27 22:58:02,740 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_mask_0010.fits'.
2021-08-27 22:58:02,741 - stpipe.Detector1Pipeline - INFO - Prefetch for PERSAT reference file is 'N/A'.
2021-08-27 22:58:02,742 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0018.fits'.
2021-08-27 22:58:02,743 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_refpix_0019.fits'.
2021-08-27 22:58:02,744 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2021-08-27 22:58:02,745 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2021-08-27 22:58:02,745 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_saturation_0027.fits'.
2021-08-27 22:58:02,746 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_superbias_0117.fits'.
2021-08-27 22:58:02,747 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPDENSITY reference file is 'N/A'.
2021-08-27 22:58:02,748 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPPARS reference file is 'N/A'.
2021-08-27 22:58:02,748 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2021-08-27 22:58:03,620 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2021-08-27 22:58:03,621 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:58:04,124 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2021-08-27 22:58:04,125 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2021-08-27 22:58:04,126 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2021-08-27 22:58:04,280 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2021-08-27 22:58:04,281 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:58:04,299 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /grp/crds/cache/references/jwst/jwst_nirspec_mask_0010.fits
2021-08-27 22:58:05,194 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2021-08-27 22:58:05,384 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2021-08-27 22:58:05,386 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:58:05,405 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /grp/crds/cache/references/jwst/jwst_nirspec_saturation_0027.fits
2021-08-27 22:58:07,221 - stpipe.Detector1Pipeline.saturation - INFO - Detected 131958 saturated pixels
2021-08-27 22:58:07,362 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2021-08-27 22:58:07,381 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2021-08-27 22:58:07,599 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2021-08-27 22:58:07,601 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:58:07,601 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2021-08-27 22:58:07,602 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc done
2021-08-27 22:58:07,757 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2021-08-27 22:58:07,758 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:58:07,780 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /grp/crds/cache/references/jwst/jwst_nirspec_superbias_0117.fits
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/stdatamodels/validate.py:34: ValidationWarning: While validating meta.exposure.readpatt the following error occurred:
'ALLIRS2' is not one of ['ACQ1', 'ACQ2', 'BRIGHT1', 'BRIGHT2', 'DEEP2', 'DEEP8', 'FAST', 'FASTGRPAVG', 'FASTGRPAVG8', 'FASTGRPAVG16', 'FASTGRPAVG32', 'FASTGRPAVG64', 'FASTR1', 'FASTR100', 'FGS', 'FGS60', 'FGS8370', 'FGS840', 'FGSRAPID', 'FINEGUIDE', 'ID', 'MEDIUM2', 'MEDIUM8', 'NIS', 'NISRAPID', 'NRS', 'NRSIRS2', 'NRSN16R4', 'NRSN32R8', 'NRSN8R2', 'NRSRAPID', 'NRSIRS2RAPID', 'NRSRAPIDD1', 'NRSRAPIDD2', 'NRSRAPIDD6', 'NRSSLOW', 'RAPID', 'SHALLOW2', 'SHALLOW4', 'SLOW', 'SLOWR1', 'TRACK', 'ANY', 'N/A']
Failed validating 'enum' in schema:
OrderedDict([('title', 'Readout pattern'),
('type', 'string'),
('enum',
['ACQ1',
'ACQ2',
'BRIGHT1',
'BRIGHT2',
'DEEP2',
'DEEP8',
'FAST',
'FASTGRPAVG',
'FASTGRPAVG8',
'FASTGRPAVG16',
'FASTGRPAVG32',
'FASTGRPAVG64',
'FASTR1',
'FASTR100',
'FGS',
'FGS60',
'FGS8370',
'FGS840',
'FGSRAPID',
'FINEGUIDE',
'ID',
'MEDIUM2',
'MEDIUM8',
'NIS',
'NISRAPID',
'NRS',
'NRSIRS2',
'NRSN16R4',
'NRSN32R8',
'NRSN8R2',
'NRSRAPID',
'NRSIRS2RAPID',
'NRSRAPIDD1',
'NRSRAPIDD2',
'NRSRAPIDD6',
'NRSSLOW',
'RAPID',
'SHALLOW2',
'SHALLOW4',
'SLOW',
'SLOWR1',
'TRACK',
'ANY',
'N/A']),
('fits_ke ...
warnings.warn(errmsg, ValidationWarning)
2021-08-27 22:58:08,811 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2021-08-27 22:58:09,013 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2021-08-27 22:58:09,014 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}
2021-08-27 22:58:09,034 - stpipe.Detector1Pipeline.refpix - INFO - Using refpix reference file: /grp/crds/cache/references/jwst/jwst_nirspec_refpix_0019.fits
2021-08-27 22:59:10,742 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2021-08-27 22:59:11,037 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2021-08-27 22:59:11,039 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 22:59:11,062 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /grp/crds/cache/references/jwst/jwst_nirspec_linearity_0022.fits
2021-08-27 22:59:11,171 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword BAD_LIN_CORR does not correspond to an existing DQ mnemonic, so will be ignored
2021-08-27 22:59:12,784 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2021-08-27 22:59:12,995 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2021-08-27 22:59:12,996 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'dark_output': None}
2021-08-27 22:59:13,187 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /grp/crds/cache/references/jwst/jwst_nirspec_dark_0088.fits
2021-08-27 22:59:16,958 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=20, nframes=1, groupgap=0
2021-08-27 22:59:16,958 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=65, nframes=1, groupgap=0
2021-08-27 22:59:18,086 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2021-08-27 22:59:18,653 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2021-08-27 22:59:18,655 - stpipe.Detector1Pipeline.jump - INFO - Step jump parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}
2021-08-27 22:59:18,751 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2021-08-27 22:59:18,852 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0023.fits
2021-08-27 22:59:19,076 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0018.fits
2021-08-27 22:59:19,121 - stpipe.Detector1Pipeline.jump - INFO - Using 1 core for jump detection
2021-08-27 22:59:19,979 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2021-08-27 22:59:20,644 - stpipe.Detector1Pipeline.jump - INFO - Working on integration 1:
2021-08-27 22:59:25,275 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 96703 pixels with at least one CR and at least four groups
2021-08-27 22:59:25,276 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 406 pixels with at least one CR and three groups
2021-08-27 22:59:25,277 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 27166 pixels with at least one CR and two groups
2021-08-27 22:59:32,947 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 12.9667 sec
2021-08-27 22:59:32,951 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 14.200008
2021-08-27 22:59:32,955 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2021-08-27 22:59:33,226 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2021-08-27 22:59:33,227 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}
2021-08-27 22:59:33,576 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0018.fits
2021-08-27 22:59:33,621 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0023.fits
2021-08-27 22:59:33,643 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = ols
2021-08-27 22:59:33,644 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/jwst/ramp_fitting/ols_fit.py:1169: RuntimeWarning: invalid value encountered in multiply
var_p4[num_int, :, :, :] *= (segs_4[num_int, :, :, :] > 0)
2021-08-27 23:02:08,291 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of groups per integration: 20
2021-08-27 23:02:08,292 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of integrations: 1
2021-08-27 23:02:08,547 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2021-08-27 23:02:08,876 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2021-08-27 23:02:08,877 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scale', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 23:02:08,929 - stpipe.Detector1Pipeline.gain_scale - INFO - Rescaling by 1.0
2021-08-27 23:02:08,941 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 23:02:09,098 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2021-08-27 23:02:09,100 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scaleints', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 23:02:09,156 - stpipe.Detector1Pipeline.gain_scale - INFO - Rescaling by 1.0
2021-08-27 23:02:09,168 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 23:02:09,169 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2021-08-27 23:02:09,170 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2021-08-27 23:02:09,179 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 23:02:09,425 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2021-08-27 23:02:09,427 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.25, 'sip_degree': None, 'sip_max_inv_pix_error': 0.25, 'sip_inv_degree': None, 'sip_npoints': 32, 'slit_y_low': -0.55, 'slit_y_high': 0.55}
2021-08-27 23:02:09,604 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.1448970586061478 deg
2021-08-27 23:02:09,605 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3232757747173309 deg
2021-08-27 23:02:09,606 - stpipe.AssignWcsStep - INFO - theta_y correction: -3.766823889842878e-13 deg
2021-08-27 23:02:09,607 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 23:02:12,986 - stpipe.AssignWcsStep - INFO - Created a NIRSPEC nrs_ifu pipeline with references {'distortion': None, 'filteroffset': None, 'specwcs': None, 'regions': None, 'wavelengthrange': '/grp/crds/cache/references/jwst/jwst_nirspec_wavelengthrange_0004.asdf', 'camera': '/grp/crds/cache/references/jwst/jwst_nirspec_camera_0004.asdf', 'collimator': '/grp/crds/cache/references/jwst/jwst_nirspec_collimator_0004.asdf', 'disperser': '/grp/crds/cache/references/jwst/jwst_nirspec_disperser_0035.asdf', 'fore': '/grp/crds/cache/references/jwst/jwst_nirspec_fore_0022.asdf', 'fpa': '/grp/crds/cache/references/jwst/jwst_nirspec_fpa_0005.asdf', 'msa': '/grp/crds/cache/references/jwst/jwst_nirspec_msa_0005.asdf', 'ote': '/grp/crds/cache/references/jwst/jwst_nirspec_ote_0005.asdf', 'ifupost': '/grp/crds/cache/references/jwst/jwst_nirspec_ifupost_0004.asdf', 'ifufore': '/grp/crds/cache/references/jwst/jwst_nirspec_ifufore_0003.asdf', 'ifuslicer': '/grp/crds/cache/references/jwst/jwst_nirspec_ifuslicer_0003.asdf'}
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/gwcs/geometry.py:203: RuntimeWarning: invalid value encountered in remainder
lon = np.mod(lon, 360.0 * u.deg if nquant else 360.0)
2021-08-27 23:02:19,583 - stpipe.AssignWcsStep - INFO - Update S_REGION to POLYGON ICRS 156.176999392 -45.687618156 156.178775679 -45.687618156 156.178775679 -45.686331563 156.176999392 -45.686331563
2021-08-27 23:02:19,585 - stpipe.AssignWcsStep - INFO - COMPLETED assign_wcs
2021-08-27 23:02:19,592 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep done
2021-08-27 23:02:19,602 - stpipe.MSAFlagOpenStep - INFO - MSAFlagOpenStep instance created.
2021-08-27 23:02:20,004 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2021-08-27 23:02:20,006 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}
2021-08-27 23:02:20,024 - stpipe.MSAFlagOpenStep - INFO - Using reference file /grp/crds/cache/references/jwst/jwst_nirspec_msaoper_0001.json
2021-08-27 23:02:20,026 - stpipe.JwstStep - INFO - JwstStep instance created.
2021-08-27 23:02:20,142 - stpipe.MSAFlagOpenStep - INFO - gwa_ytilt is 0.1448970586061478 deg
2021-08-27 23:02:20,143 - stpipe.MSAFlagOpenStep - INFO - gwa_xtilt is 0.3232757747173309 deg
2021-08-27 23:02:20,143 - stpipe.MSAFlagOpenStep - INFO - theta_y correction: -3.766823889842878e-13 deg
2021-08-27 23:02:20,144 - stpipe.MSAFlagOpenStep - INFO - theta_x correction: 0.0 deg
2021-08-27 23:02:20,152 - stpipe.MSAFlagOpenStep - INFO - SPORDER= -1, wrange=[2.87e-06, 5.27e-06]
2021-08-27 23:02:20,265 - stpipe.MSAFlagOpenStep - INFO - There are 5 open slits in quadrant 1
2021-08-27 23:02:20,304 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 2
2021-08-27 23:02:20,329 - stpipe.MSAFlagOpenStep - INFO - There are 9 open slits in quadrant 3
2021-08-27 23:02:20,401 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 4
2021-08-27 23:02:20,426 - stpipe.MSAFlagOpenStep - INFO - There are 0 open slits in quadrant 5
2021-08-27 23:02:33,569 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep done
2021-08-27 23:02:34,245 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 23:02:34,257 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 23:02:35,170 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2021-08-27 23:02:35,172 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.25, 'sip_degree': None, 'sip_max_inv_pix_error': 0.25, 'sip_inv_degree': None, 'sip_npoints': 32, 'slit_y_low': -0.45, 'slit_y_high': 0.45}
2021-08-27 23:02:35,331 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.1448970586061478 deg
2021-08-27 23:02:35,332 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3232757747173309 deg
2021-08-27 23:02:35,332 - stpipe.AssignWcsStep - INFO - theta_y correction: -3.766823889842878e-13 deg
2021-08-27 23:02:35,333 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 23:02:38,706 - stpipe.AssignWcsStep - INFO - Created a NIRSPEC nrs_ifu pipeline with references {'distortion': None, 'filteroffset': None, 'specwcs': None, 'regions': None, 'wavelengthrange': '/grp/crds/cache/references/jwst/jwst_nirspec_wavelengthrange_0004.asdf', 'camera': '/grp/crds/cache/references/jwst/jwst_nirspec_camera_0004.asdf', 'collimator': '/grp/crds/cache/references/jwst/jwst_nirspec_collimator_0004.asdf', 'disperser': '/grp/crds/cache/references/jwst/jwst_nirspec_disperser_0035.asdf', 'fore': '/grp/crds/cache/references/jwst/jwst_nirspec_fore_0022.asdf', 'fpa': '/grp/crds/cache/references/jwst/jwst_nirspec_fpa_0005.asdf', 'msa': '/grp/crds/cache/references/jwst/jwst_nirspec_msa_0005.asdf', 'ote': '/grp/crds/cache/references/jwst/jwst_nirspec_ote_0005.asdf', 'ifupost': '/grp/crds/cache/references/jwst/jwst_nirspec_ifupost_0004.asdf', 'ifufore': '/grp/crds/cache/references/jwst/jwst_nirspec_ifufore_0003.asdf', 'ifuslicer': '/grp/crds/cache/references/jwst/jwst_nirspec_ifuslicer_0003.asdf'}
2021-08-27 23:02:45,855 - stpipe.AssignWcsStep - INFO - Update S_REGION to POLYGON ICRS 156.177073911 -45.687546255 156.178701937 -45.687546255 156.178701937 -45.686403050 156.177073911 -45.686403050
2021-08-27 23:02:45,857 - stpipe.AssignWcsStep - INFO - COMPLETED assign_wcs
2021-08-27 23:02:45,863 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep done
Working with slit/slice: 0 bounding box rounded to next integer: [[ 274 2048] [ 780 830]] Max value in slity array (ignoring NANs): 0.449990345211875 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 1 bounding box rounded to next integer: [[ 311 2048] [1196 1248]] Max value in slity array (ignoring NANs): 0.4499957907513946 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 2 bounding box rounded to next integer: [[ 270 2048] [ 730 781]] Max value in slity array (ignoring NANs): 0.44998649767873244 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 3 bounding box rounded to next integer: [[ 316 2048] [1245 1297]] Max value in slity array (ignoring NANs): 0.44998302744762897 percentage of F/O trace that was flagged: 17.458563535911605
Working with slit/slice: 4 bounding box rounded to next integer: [[ 265 2048] [ 681 732]] Max value in slity array (ignoring NANs): 0.4499951916396593 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 5 bounding box rounded to next integer: [[ 320 2048] [1295 1346]] Max value in slity array (ignoring NANs): 0.4499773521043404 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 6 bounding box rounded to next integer: [[ 261 2048] [ 632 682]] Max value in slity array (ignoring NANs): 0.44999427828406297 percentage of F/O trace that was flagged: 14.704781704781706
Working with slit/slice: 7 bounding box rounded to next integer: [[ 325 2048] [1344 1395]] Max value in slity array (ignoring NANs): 0.44999604029909124 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 8 bounding box rounded to next integer: [[ 257 2048] [ 583 633]] Max value in slity array (ignoring NANs): 0.4499700328353698 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 9 bounding box rounded to next integer: [[ 330 2048] [1393 1445]] Max value in slity array (ignoring NANs): 0.44998733429292376 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 10 bounding box rounded to next integer: [[ 253 2048] [ 533 584]] Max value in slity array (ignoring NANs): 0.44997951314134865 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 11 bounding box rounded to next integer: [[ 335 2048] [1441 1494]] Max value in slity array (ignoring NANs): 0.44998216614061454 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 12 bounding box rounded to next integer: [[ 249 2048] [ 484 534]] Max value in slity array (ignoring NANs): 0.4499628763917655 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 13 bounding box rounded to next integer: [[ 339 2048] [1490 1543]] Max value in slity array (ignoring NANs): 0.4499732254076479 percentage of F/O trace that was flagged: 4.141178541565814
Working with slit/slice: 14 bounding box rounded to next integer: [[ 245 2048] [ 435 485]] Max value in slity array (ignoring NANs): 0.44999164853188245 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 15 bounding box rounded to next integer: [[ 344 2048] [1539 1592]] Max value in slity array (ignoring NANs): 0.44998852380040494 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 16 bounding box rounded to next integer: [[ 241 2048] [ 385 436]] Max value in slity array (ignoring NANs): 0.4499870779821324 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 17 bounding box rounded to next integer: [[ 349 2048] [1588 1641]] Max value in slity array (ignoring NANs): 0.4499705022522089 percentage of F/O trace that was flagged: 3.2492650471917064
Working with slit/slice: 18 bounding box rounded to next integer: [[ 237 2048] [ 336 387]] Max value in slity array (ignoring NANs): 0.4499898319356532 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 19 bounding box rounded to next integer: [[ 354 2048] [1637 1690]] Max value in slity array (ignoring NANs): 0.44999842902952475 percentage of F/O trace that was flagged: 0.8359642491884922
Working with slit/slice: 20 bounding box rounded to next integer: [[ 234 2048] [ 287 337]] Max value in slity array (ignoring NANs): 0.44998944287684794 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 21 bounding box rounded to next integer: [[ 359 2048] [1686 1739]] Max value in slity array (ignoring NANs): 0.449982979106301 percentage of F/O trace that was flagged: 11.892374679021994
Working with slit/slice: 22 bounding box rounded to next integer: [[ 230 2048] [ 237 288]] Max value in slity array (ignoring NANs): 0.4499873120559427 percentage of F/O trace that was flagged: 3.322507111012487
Working with slit/slice: 23 bounding box rounded to next integer: [[ 364 2048] [1735 1789]] Max value in slity array (ignoring NANs): 0.4499801506771266 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 24 bounding box rounded to next integer: [[ 226 2048] [ 188 238]] Max value in slity array (ignoring NANs): 0.4499948696973963 percentage of F/O trace that was flagged: 13.881048185294176
Working with slit/slice: 25 bounding box rounded to next integer: [[ 369 2048] [1783 1838]] Max value in slity array (ignoring NANs): 0.4499751505794029 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 26 bounding box rounded to next integer: [[ 222 2048] [ 139 189]] Max value in slity array (ignoring NANs): 0.4499967307983039 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 27 bounding box rounded to next integer: [[ 374 2048] [1832 1887]] Max value in slity array (ignoring NANs): 0.44999133942399405 percentage of F/O trace that was flagged: 2.163439891940567
Working with slit/slice: 28 bounding box rounded to next integer: [[ 218 2048] [ 89 139]] Max value in slity array (ignoring NANs): 0.44999837058851305 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 29 bounding box rounded to next integer: [[ 379 2048] [1881 1936]] Max value in slity array (ignoring NANs): 0.44999782023370255 percentage of F/O trace that was flagged: 0.0
*** Final result for msa_flagging test will be reported as FAILED ***
('* MSA flagging validation test took ', '1.5967050830523173 minutes to finish.')
Did msa_flagging validation test pass? False
Testing files for detector: nrs2
Working with uncal_file: /data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/ifu_g395h_f290lp_nrs2_uncal.fits
2021-08-27 23:04:12,248 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /grp/crds/cache/references/jwst/jwst_nirspec_pars-detector1pipeline_0004.asdf
2021-08-27 23:04:12,262 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2021-08-27 23:04:12,263 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2021-08-27 23:04:12,264 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2021-08-27 23:04:12,266 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2021-08-27 23:04:12,267 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2021-08-27 23:04:12,267 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2021-08-27 23:04:12,268 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2021-08-27 23:04:12,269 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2021-08-27 23:04:12,270 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2021-08-27 23:04:12,271 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2021-08-27 23:04:12,273 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2021-08-27 23:04:12,274 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2021-08-27 23:04:12,275 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2021-08-27 23:04:12,276 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2021-08-27 23:04:12,277 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2021-08-27 23:04:12,278 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2021-08-27 23:04:12,279 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/stpipe/step.py:359: ResourceWarning: unclosed file <_io.FileIO name='/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/ifu_g395h_f290lp_nrs2_uncal.fits' mode='rb' closefd=True>
gc.collect()
2021-08-27 23:04:13,145 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7/ifu_g395h_f290lp_nrs2_uncal.fits',).
2021-08-27 23:04:13,156 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_calibrated_ramp': False, 'steps': {'group_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dq_init': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'saturation': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'ipc': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'superbias': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'refpix': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}, 'rscd': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'type': 'baseline'}, 'firstframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'lastframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'linearity': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dark_current': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'dark_output': None}, 'reset': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'persistence': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'input_trapsfilled': '', 'flag_pers_cutoff': 40.0, 'save_persistence': False, 'save_trapsfilled': True}, 'jump': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}, 'ramp_fit': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}, 'gain_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}}}
2021-08-27 23:04:13,438 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'ifu_g395h_f290lp_nrs2_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'persat', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias', 'trapdensity', 'trappars']
2021-08-27 23:04:13,448 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_dark_0085.fits'.
2021-08-27 23:04:13,449 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_gain_0020.fits'.
2021-08-27 23:04:13,451 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_linearity_0025.fits'.
2021-08-27 23:04:13,452 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_mask_0011.fits'.
2021-08-27 23:04:13,454 - stpipe.Detector1Pipeline - INFO - Prefetch for PERSAT reference file is 'N/A'.
2021-08-27 23:04:13,454 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0017.fits'.
2021-08-27 23:04:13,455 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_refpix_0018.fits'.
2021-08-27 23:04:13,456 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2021-08-27 23:04:13,456 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2021-08-27 23:04:13,457 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_saturation_0026.fits'.
2021-08-27 23:04:13,458 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_superbias_0118.fits'.
2021-08-27 23:04:13,459 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPDENSITY reference file is 'N/A'.
2021-08-27 23:04:13,459 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPPARS reference file is 'N/A'.
2021-08-27 23:04:13,460 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2021-08-27 23:04:14,487 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2021-08-27 23:04:14,489 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 23:04:14,939 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2021-08-27 23:04:14,940 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2021-08-27 23:04:14,941 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2021-08-27 23:04:15,255 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2021-08-27 23:04:15,257 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 23:04:15,277 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /grp/crds/cache/references/jwst/jwst_nirspec_mask_0011.fits
2021-08-27 23:04:16,272 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2021-08-27 23:04:16,602 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2021-08-27 23:04:16,604 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 23:04:16,623 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /grp/crds/cache/references/jwst/jwst_nirspec_saturation_0026.fits
2021-08-27 23:04:18,924 - stpipe.Detector1Pipeline.saturation - INFO - Detected 151502 saturated pixels
2021-08-27 23:04:19,119 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2021-08-27 23:04:19,149 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2021-08-27 23:04:19,539 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2021-08-27 23:04:19,541 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 23:04:19,542 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2021-08-27 23:04:19,543 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc done
2021-08-27 23:04:19,855 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2021-08-27 23:04:19,856 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 23:04:19,878 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /grp/crds/cache/references/jwst/jwst_nirspec_superbias_0118.fits
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/stdatamodels/validate.py:34: ValidationWarning: While validating meta.exposure.readpatt the following error occurred:
'ALLIRS2' is not one of ['ACQ1', 'ACQ2', 'BRIGHT1', 'BRIGHT2', 'DEEP2', 'DEEP8', 'FAST', 'FASTGRPAVG', 'FASTGRPAVG8', 'FASTGRPAVG16', 'FASTGRPAVG32', 'FASTGRPAVG64', 'FASTR1', 'FASTR100', 'FGS', 'FGS60', 'FGS8370', 'FGS840', 'FGSRAPID', 'FINEGUIDE', 'ID', 'MEDIUM2', 'MEDIUM8', 'NIS', 'NISRAPID', 'NRS', 'NRSIRS2', 'NRSN16R4', 'NRSN32R8', 'NRSN8R2', 'NRSRAPID', 'NRSIRS2RAPID', 'NRSRAPIDD1', 'NRSRAPIDD2', 'NRSRAPIDD6', 'NRSSLOW', 'RAPID', 'SHALLOW2', 'SHALLOW4', 'SLOW', 'SLOWR1', 'TRACK', 'ANY', 'N/A']
Failed validating 'enum' in schema:
OrderedDict([('title', 'Readout pattern'),
('type', 'string'),
('enum',
['ACQ1',
'ACQ2',
'BRIGHT1',
'BRIGHT2',
'DEEP2',
'DEEP8',
'FAST',
'FASTGRPAVG',
'FASTGRPAVG8',
'FASTGRPAVG16',
'FASTGRPAVG32',
'FASTGRPAVG64',
'FASTR1',
'FASTR100',
'FGS',
'FGS60',
'FGS8370',
'FGS840',
'FGSRAPID',
'FINEGUIDE',
'ID',
'MEDIUM2',
'MEDIUM8',
'NIS',
'NISRAPID',
'NRS',
'NRSIRS2',
'NRSN16R4',
'NRSN32R8',
'NRSN8R2',
'NRSRAPID',
'NRSIRS2RAPID',
'NRSRAPIDD1',
'NRSRAPIDD2',
'NRSRAPIDD6',
'NRSSLOW',
'RAPID',
'SHALLOW2',
'SHALLOW4',
'SLOW',
'SLOWR1',
'TRACK',
'ANY',
'N/A']),
('fits_ke ...
warnings.warn(errmsg, ValidationWarning)
2021-08-27 23:04:20,988 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2021-08-27 23:04:21,319 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2021-08-27 23:04:21,321 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}
2021-08-27 23:04:21,340 - stpipe.Detector1Pipeline.refpix - INFO - Using refpix reference file: /grp/crds/cache/references/jwst/jwst_nirspec_refpix_0018.fits
2021-08-27 23:05:25,697 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2021-08-27 23:05:26,056 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2021-08-27 23:05:26,058 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 23:05:26,082 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /grp/crds/cache/references/jwst/jwst_nirspec_linearity_0025.fits
2021-08-27 23:05:26,195 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword BAD_LIN_CORR does not correspond to an existing DQ mnemonic, so will be ignored
2021-08-27 23:05:27,975 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2021-08-27 23:05:28,162 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2021-08-27 23:05:28,164 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'dark_output': None}
2021-08-27 23:05:28,574 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /grp/crds/cache/references/jwst/jwst_nirspec_dark_0085.fits
2021-08-27 23:05:32,215 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=20, nframes=1, groupgap=0
2021-08-27 23:05:32,216 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=65, nframes=1, groupgap=0
2021-08-27 23:05:33,302 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2021-08-27 23:05:33,920 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2021-08-27 23:05:33,922 - stpipe.Detector1Pipeline.jump - INFO - Step jump parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0}
2021-08-27 23:05:34,126 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2021-08-27 23:05:34,334 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0020.fits
2021-08-27 23:05:34,558 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0017.fits
2021-08-27 23:05:34,601 - stpipe.Detector1Pipeline.jump - INFO - Using 1 core for jump detection
2021-08-27 23:05:35,436 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2021-08-27 23:05:36,096 - stpipe.Detector1Pipeline.jump - INFO - Working on integration 1:
2021-08-27 23:05:40,665 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 76942 pixels with at least one CR and at least four groups
2021-08-27 23:05:40,666 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 547 pixels with at least one CR and three groups
2021-08-27 23:05:40,667 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier Two-point found 25384 pixels with at least one CR and two groups
2021-08-27 23:05:49,075 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 13.638 sec
2021-08-27 23:05:49,078 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 14.952142
2021-08-27 23:05:49,081 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2021-08-27 23:05:49,363 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2021-08-27 23:05:49,365 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7', 'int_name': '', 'save_opt': False, 'opt_name': '', 'maximum_cores': 'none'}
2021-08-27 23:05:49,923 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0017.fits
2021-08-27 23:05:49,964 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0020.fits
2021-08-27 23:05:49,985 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = ols
2021-08-27 23:05:49,985 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/jwst/ramp_fitting/ols_fit.py:1169: RuntimeWarning: invalid value encountered in multiply
var_p4[num_int, :, :, :] *= (segs_4[num_int, :, :, :] > 0)
2021-08-27 23:08:26,126 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of groups per integration: 20
2021-08-27 23:08:26,127 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of integrations: 1
2021-08-27 23:08:26,383 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2021-08-27 23:08:26,756 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2021-08-27 23:08:26,757 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scale', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 23:08:26,808 - stpipe.Detector1Pipeline.gain_scale - INFO - Rescaling by 1.0
2021-08-27 23:08:26,819 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 23:08:26,976 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2021-08-27 23:08:26,977 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scaleints', 'search_output_file': True, 'input_dir': '/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmpm79hihb7'}
2021-08-27 23:08:27,042 - stpipe.Detector1Pipeline.gain_scale - INFO - Rescaling by 1.0
2021-08-27 23:08:27,053 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2021-08-27 23:08:27,054 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2021-08-27 23:08:27,055 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2021-08-27 23:08:27,065 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 23:08:27,231 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2021-08-27 23:08:27,232 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.25, 'sip_degree': None, 'sip_max_inv_pix_error': 0.25, 'sip_inv_degree': None, 'sip_npoints': 32, 'slit_y_low': -0.55, 'slit_y_high': 0.55}
2021-08-27 23:08:27,412 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.1448970586061478 deg
2021-08-27 23:08:27,412 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3232757747173309 deg
2021-08-27 23:08:27,413 - stpipe.AssignWcsStep - INFO - theta_y correction: -3.766823889842878e-13 deg
2021-08-27 23:08:27,414 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 23:08:30,761 - stpipe.AssignWcsStep - INFO - Created a NIRSPEC nrs_ifu pipeline with references {'distortion': None, 'filteroffset': None, 'specwcs': None, 'regions': None, 'wavelengthrange': '/grp/crds/cache/references/jwst/jwst_nirspec_wavelengthrange_0004.asdf', 'camera': '/grp/crds/cache/references/jwst/jwst_nirspec_camera_0004.asdf', 'collimator': '/grp/crds/cache/references/jwst/jwst_nirspec_collimator_0004.asdf', 'disperser': '/grp/crds/cache/references/jwst/jwst_nirspec_disperser_0035.asdf', 'fore': '/grp/crds/cache/references/jwst/jwst_nirspec_fore_0022.asdf', 'fpa': '/grp/crds/cache/references/jwst/jwst_nirspec_fpa_0005.asdf', 'msa': '/grp/crds/cache/references/jwst/jwst_nirspec_msa_0005.asdf', 'ote': '/grp/crds/cache/references/jwst/jwst_nirspec_ote_0005.asdf', 'ifupost': '/grp/crds/cache/references/jwst/jwst_nirspec_ifupost_0004.asdf', 'ifufore': '/grp/crds/cache/references/jwst/jwst_nirspec_ifufore_0003.asdf', 'ifuslicer': '/grp/crds/cache/references/jwst/jwst_nirspec_ifuslicer_0003.asdf'}
/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.8/site-packages/gwcs/geometry.py:203: RuntimeWarning: invalid value encountered in remainder
lon = np.mod(lon, 360.0 * u.deg if nquant else 360.0)
2021-08-27 23:08:37,397 - stpipe.AssignWcsStep - INFO - Update S_REGION to POLYGON ICRS 156.176996289 -45.687617183 156.178772625 -45.687617183 156.178772625 -45.686330364 156.176996289 -45.686330364
2021-08-27 23:08:37,399 - stpipe.AssignWcsStep - INFO - COMPLETED assign_wcs
2021-08-27 23:08:37,405 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep done
2021-08-27 23:08:37,415 - stpipe.MSAFlagOpenStep - INFO - MSAFlagOpenStep instance created.
2021-08-27 23:08:37,748 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2021-08-27 23:08:37,750 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}
2021-08-27 23:08:37,769 - stpipe.MSAFlagOpenStep - INFO - Using reference file /grp/crds/cache/references/jwst/jwst_nirspec_msaoper_0001.json
2021-08-27 23:08:37,770 - stpipe.JwstStep - INFO - JwstStep instance created.
2021-08-27 23:08:37,886 - stpipe.MSAFlagOpenStep - INFO - gwa_ytilt is 0.1448970586061478 deg
2021-08-27 23:08:37,887 - stpipe.MSAFlagOpenStep - INFO - gwa_xtilt is 0.3232757747173309 deg
2021-08-27 23:08:37,887 - stpipe.MSAFlagOpenStep - INFO - theta_y correction: -3.766823889842878e-13 deg
2021-08-27 23:08:37,888 - stpipe.MSAFlagOpenStep - INFO - theta_x correction: 0.0 deg
2021-08-27 23:08:37,897 - stpipe.MSAFlagOpenStep - INFO - SPORDER= -1, wrange=[2.87e-06, 5.27e-06]
2021-08-27 23:08:38,007 - stpipe.MSAFlagOpenStep - INFO - There are 5 open slits in quadrant 1
2021-08-27 23:08:38,046 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 2
2021-08-27 23:08:38,071 - stpipe.MSAFlagOpenStep - INFO - There are 9 open slits in quadrant 3
2021-08-27 23:08:38,142 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 4
2021-08-27 23:08:38,166 - stpipe.MSAFlagOpenStep - INFO - There are 0 open slits in quadrant 5
2021-08-27 23:08:52,259 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep done
2021-08-27 23:08:52,939 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 23:08:52,951 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2021-08-27 23:08:53,835 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2021-08-27 23:08:53,838 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.25, 'sip_degree': None, 'sip_max_inv_pix_error': 0.25, 'sip_inv_degree': None, 'sip_npoints': 32, 'slit_y_low': -0.45, 'slit_y_high': 0.45}
2021-08-27 23:08:53,995 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.1448970586061478 deg
2021-08-27 23:08:53,996 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3232757747173309 deg
2021-08-27 23:08:53,996 - stpipe.AssignWcsStep - INFO - theta_y correction: -3.766823889842878e-13 deg
2021-08-27 23:08:53,997 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2021-08-27 23:08:57,295 - stpipe.AssignWcsStep - INFO - Created a NIRSPEC nrs_ifu pipeline with references {'distortion': None, 'filteroffset': None, 'specwcs': None, 'regions': None, 'wavelengthrange': '/grp/crds/cache/references/jwst/jwst_nirspec_wavelengthrange_0004.asdf', 'camera': '/grp/crds/cache/references/jwst/jwst_nirspec_camera_0004.asdf', 'collimator': '/grp/crds/cache/references/jwst/jwst_nirspec_collimator_0004.asdf', 'disperser': '/grp/crds/cache/references/jwst/jwst_nirspec_disperser_0035.asdf', 'fore': '/grp/crds/cache/references/jwst/jwst_nirspec_fore_0022.asdf', 'fpa': '/grp/crds/cache/references/jwst/jwst_nirspec_fpa_0005.asdf', 'msa': '/grp/crds/cache/references/jwst/jwst_nirspec_msa_0005.asdf', 'ote': '/grp/crds/cache/references/jwst/jwst_nirspec_ote_0005.asdf', 'ifupost': '/grp/crds/cache/references/jwst/jwst_nirspec_ifupost_0004.asdf', 'ifufore': '/grp/crds/cache/references/jwst/jwst_nirspec_ifufore_0003.asdf', 'ifuslicer': '/grp/crds/cache/references/jwst/jwst_nirspec_ifuslicer_0003.asdf'}
2021-08-27 23:09:03,811 - stpipe.AssignWcsStep - INFO - Update S_REGION to POLYGON ICRS 156.177070769 -45.687545142 156.178698884 -45.687545142 156.178698884 -45.686401970 156.177070769 -45.686401970
2021-08-27 23:09:03,813 - stpipe.AssignWcsStep - INFO - COMPLETED assign_wcs
2021-08-27 23:09:03,819 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep done
Working with slit/slice: 0 bounding box rounded to next integer: [[ 0 1676] [ 780 830]] Max value in slity array (ignoring NANs): 0.4499698976797671 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 1 bounding box rounded to next integer: [[ 0 1721] [1196 1248]] Max value in slity array (ignoring NANs): 0.44999474521530664 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 2 bounding box rounded to next integer: [[ 0 1671] [ 730 781]] Max value in slity array (ignoring NANs): 0.449993208905082 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 3 bounding box rounded to next integer: [[ 0 1726] [1246 1297]] Max value in slity array (ignoring NANs): 0.4499917562865504 percentage of F/O trace that was flagged: 32.66585408323176
Working with slit/slice: 4 bounding box rounded to next integer: [[ 0 1666] [ 681 732]] Max value in slity array (ignoring NANs): 0.4499977553638593 percentage of F/O trace that was flagged: 14.232410572647854
Working with slit/slice: 5 bounding box rounded to next integer: [[ 0 1731] [1295 1346]] Max value in slity array (ignoring NANs): 0.4499942268926361 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 6 bounding box rounded to next integer: [[ 0 1662] [ 632 682]] Max value in slity array (ignoring NANs): 0.4499536072866851 percentage of F/O trace that was flagged: 16.16992733370598
Working with slit/slice: 7 bounding box rounded to next integer: [[ 0 1737] [1344 1395]] Max value in slity array (ignoring NANs): 0.44998115931737176 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 8 bounding box rounded to next integer: [[ 0 1657] [ 583 633]] Max value in slity array (ignoring NANs): 0.44997713770634123 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 9 bounding box rounded to next integer: [[ 0 1742] [1393 1445]] Max value in slity array (ignoring NANs): 0.44999791822619517 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 10 bounding box rounded to next integer: [[ 0 1652] [ 533 584]] Max value in slity array (ignoring NANs): 0.4499968565007051 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 11 bounding box rounded to next integer: [[ 0 1748] [1442 1494]] Max value in slity array (ignoring NANs): 0.4499972292014403 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 12 bounding box rounded to next integer: [[ 0 1647] [ 484 535]] Max value in slity array (ignoring NANs): 0.44999910431927576 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 13 bounding box rounded to next integer: [[ 0 1754] [1490 1543]] Max value in slity array (ignoring NANs): 0.4499987438708725 percentage of F/O trace that was flagged: 4.809817003840131
Working with slit/slice: 14 bounding box rounded to next integer: [[ 0 1643] [ 435 485]] Max value in slity array (ignoring NANs): 0.4499951255331078 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 15 bounding box rounded to next integer: [[ 0 1760] [1539 1592]] Max value in slity array (ignoring NANs): 0.44999518183787895 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 16 bounding box rounded to next integer: [[ 0 1638] [ 386 436]] Max value in slity array (ignoring NANs): 0.44999925976278876 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 17 bounding box rounded to next integer: [[ 0 1766] [1588 1641]] Max value in slity array (ignoring NANs): 0.4499888190799727 percentage of F/O trace that was flagged: 16.313807264293466
Working with slit/slice: 18 bounding box rounded to next integer: [[ 0 1634] [ 336 387]] Max value in slity array (ignoring NANs): 0.4499804732626374 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 19 bounding box rounded to next integer: [[ 0 1771] [1637 1690]] Max value in slity array (ignoring NANs): 0.449994489025675 percentage of F/O trace that was flagged: 8.611453594660091
Working with slit/slice: 20 bounding box rounded to next integer: [[ 0 1629] [ 287 337]] Max value in slity array (ignoring NANs): 0.44999763808730675 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 21 bounding box rounded to next integer: [[ 0 1777] [1686 1740]] Max value in slity array (ignoring NANs): 0.44999682555028364 percentage of F/O trace that was flagged: 16.13703860840051
Working with slit/slice: 22 bounding box rounded to next integer: [[ 0 1625] [ 238 288]] Max value in slity array (ignoring NANs): 0.44999728382322296 percentage of F/O trace that was flagged: 15.730058025332461
Working with slit/slice: 23 bounding box rounded to next integer: [[ 0 1783] [1735 1789]] Max value in slity array (ignoring NANs): 0.44997001701461925 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 24 bounding box rounded to next integer: [[ 0 1620] [ 188 239]] Max value in slity array (ignoring NANs): 0.4499844496052553 percentage of F/O trace that was flagged: 15.801567991349014
Working with slit/slice: 25 bounding box rounded to next integer: [[ 0 1789] [1783 1838]] Max value in slity array (ignoring NANs): 0.4499975610566848 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 26 bounding box rounded to next integer: [[ 0 1616] [ 139 189]] Max value in slity array (ignoring NANs): 0.44998412011574584 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 27 bounding box rounded to next integer: [[ 0 1796] [1832 1887]] Max value in slity array (ignoring NANs): 0.44997797852583443 percentage of F/O trace that was flagged: 3.7213300348022975
Working with slit/slice: 28 bounding box rounded to next integer: [[ 0 1612] [ 89 139]] Max value in slity array (ignoring NANs): 0.44996492684454875 percentage of F/O trace that was flagged: 0.0
Working with slit/slice: 29 bounding box rounded to next integer: [[ 0 1802] [1881 1936]] Max value in slity array (ignoring NANs): 0.44996718010021697 percentage of F/O trace that was flagged: 0.0
*** Final result for msa_flagging test will be reported as FAILED ***
('* MSA flagging validation test took ', '1.5461623589197795 minutes to finish.')
Did msa_flagging validation test pass? False
# Quickly see if the test passed: Do the NIRSpec implementation and the pipeline's agree within <= 99.5%?
print('These are the final results of the tests: ')
for key, val in results_dict.items():
if not isinstance(val, str):
if val:
val = 'PASSED'
else:
val = 'FAILED'
print('{:<40} {:<8}'.format(key, val))
These are the final results of the tests: mos_prism_nrs1_uncal.fits FAILED mos_prism_nrs2_uncal.fits skipped mos_g140m_line1_NRS1_uncal.fits PASSED mos_g140m_line1_NRS2_uncal.fits PASSED ifu_prism_nrs1_uncal.fits FAILED ifu_prism_nrs2_uncal.fits skipped ifu_g395h_f290lp_nrs1_uncal.fits FAILED ifu_g395h_f290lp_nrs2_uncal.fits FAILED